xiaoxiaosuaxuan
xiaoxiaosuaxuan
I met this error during fp16 inference, too. Does anyone know to fix it?
> I met this error during fp16 inference, too. Does anyone know to fix it? my solution: manually convert both model and inputs to torch.half ``` with torch.autocast(device_type='cuda', dtype=torch.half), torch.no_grad():...
any update?
@bhushan23 Thanks for your timely reply! Now I am able to run the context binary file with htp backend on my device. However, the generated result is incorrect: **it produces...