half precision for inference

the model is trained in bf16-mixed precision, so this patch uses bfloat16 for inference when cuda is available. empirically for the 600m model this improves inference speed ~8-9x