ecilay

@nvpohanh That's what I thought initially too, but the converted from Pytorch and onnx are both fp16. The pytorch model is the clip's [text encoder](https://github.com/openai/CLIP/blob/main/clip/model.py#L343). During model loading, it [converts...

Is there a tool we can visualize if it is overflown with fp16?

Sorry I don't think I can share this in public since this is my company's model in production. But it is just clip's text encoder, which I shared repro steps...

`File "/home/test/runtime/runtime/ait/eps_ait.py", line 485, in __call__ return self.forward(` `File "/home/test/runtime/runtime/ait/eps_ait.py", line 791, in forward noise_pred = self.dispatch_resolution_forward(inputs)` `File "/home/test/runtime/runtime/ait/eps_ait.py", line 890, in dispatch_resolution_forward cur_engines[f"{h}x{w}"].run_with_tensors(inputs, ys, graph_mode=False)` `File "/opt/conda/envs/test/lib/python3.10/site-packages/aitemplate/compiler/model.py", line 587,...

@chenyang78 I think you can repro by using any two AIT model (or maybe they could be the same model), load them on different GPUs, and do inference, see if...

same problem as above. Do you have resolved it? @xieqiangqiang @Spiritdude @YuDeng