torch2trt
torch2trt copied to clipboard
Encountered known unsupported method
I use torch2trt to convert a trained model. It output below messages.
Warning: Encountered known unsupported method torch.nn.functional.has_torch_function_unary
Warning: Encountered known unsupported method torch.nn.functional.silu
Warning: Encountered known unsupported method torch.nn.functional.has_torch_function_unary
Warning: Encountered known unsupported method torch.nn.functional.adaptive_avg_pool2d
...
However, the output of original model and converted model are the same.
with torch.no_grad():
output = model(input_data)
output_trt = model_trt(input_data)
# check the output against PyTorch
for i in range(4):
print(torch.max(torch.abs(output[i]- output_trt[i])))
tensor(0., device='cuda:0')
tensor(8.5325e-09, device='cuda:0')
tensor(1.0868e-08, device='cuda:0')
tensor(2.9802e-08, device='cuda:0')
My questions are
- What caused the warning
- Unspported op seems fatal but why torch2trt model can output the same result as original model.
- Can I use tritonserver to deploy torch2trt model, like torch_tensorrt model