🐛 [Bug] Encountered bug when compiling yolov5-face JIT model

[sriderya]

Bug Description

I try to convert yolov5-face torch model (more specifically yolov5n-0.5.pt) to TensorRT embedded torchscript model with your repo. The JIT model works correctly and there is no problem in pytorch side. When I run the converter code by your instructions, I get the following error.

  File "/home/python-ce/helpers/pydev/pydevd.py", line 1483, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/torch-tensorRT/test_torch-tensorrt.py", line 35, in <module>
    enabled_precisions={torch.half})
  File "/home//torch_tensorrt/_compile.py", line 97, in compile
    return torch_tensorrt.ts.compile(ts_mod, inputs=inputs, enabled_precisions=enabled_precisions, **kwargs)
  File "/home/.local/lib/python3.6/site-packages/torch_tensorrt/ts/_compiler.py", line 119, in compile
    compiled_cpp_mod = _C.compile_graph(module._c, _parse_compile_spec(spec))
RuntimeError: outputs_[i]->uses().empty()INTERNAL ASSERT FAILED at "../torch/csrc/jit/ir/ir.cpp":1286, please report a bug to PyTorch. 
python-BaseException

To Reproduce

You can run the following code:

import torch_tensorrt
import torch
from PIL import Image
from torchvision import transforms
import cv2
import torchvision.models as models

img = cv2.imread('zidane.jpg')
img = cv2.resize(img, (1600,960))
img = Image.fromarray(img).convert('RGB')

convert_tensor = transforms.ToTensor()
img = convert_tensor(img).unsqueeze(0).to('cuda')

jit_module = torch.jit.load("yolov5n-0.5-jit.pt").eval().to('cuda')

trt_ts_module = torch_tensorrt.compile(jit_module,
    inputs=[
        img,
    ],
    enabled_precisions={torch.half})

result = trt_ts_module(img).cpu().detach().numpy()
cv2.imwrite('results_img.jpg', result)
torch.jit.save(trt_ts_module, "trt_torchscript_module.ts") # save the TRT embedded Torchscript

Expected behavior

The code should convert Torch model to TensorRT embedded torchscript model and save the corresponding model.

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

• Torch-TensorRT Version : 1.0.0
• TensorRT Version : 8.0.3.4
• PyTorch Version : 1.10.1
• OS : CentOS 7
• How you installed PyTorch : pip
• Python version: 3.6.8
• CUDA version: 11.3
• GPU models and configuration: GeForce RTX 2060 Super

Additional context

I tried to convert pretrained ResNet50 model from torchvision, it is converted and gave no error with the same configuration, code and environment.

Thanks in advance!

[narendasan]

Can you share the debug logs?

[sriderya]

I am sorry but which debug logs that you mentioned about? Only log I have is the one in the issue.

[narendasan]

So try running your application but with torch_tensorrt.logging. set_reportable_log_level ( level : torch_tensorrt.logging.Level ) set to DEBUG

[sriderya]

First of all, thanks for your quick reply @narendasan. I tried what you said as follows:

import torch_tensorrt
import torch
from PIL import Image
from torchvision import transforms
import cv2
import torchvision.models as models

img = cv2.imread('zidane.jpg')
img = cv2.resize(img, (1600,960))
img = Image.fromarray(img).convert('RGB')

convert_tensor = transforms.ToTensor()
img = convert_tensor(img).unsqueeze(0).to('cuda')

jit_module = torch.jit.load("yolov5n-0.5-jit.pt").eval().to('cuda')

torch_tensorrt.logging.set_reportable_log_level(torch_tensorrt.logging.Level.Debug)
trt_ts_module = torch_tensorrt.compile(jit_module,
    inputs=[
        img,
    ],
    enabled_precisions={torch.half})

result = trt_ts_module(img).cpu().detach().numpy()
cv2.imwrite('results_img.jpg', result)
torch.jit.save(trt_ts_module, "trt_torchscript_module.ts") # save the TRT embedded Torchscript

Unfortunately, the log gives no clue:

INFO: [Torch-TensorRT] - ir was set to default, using TorchScript as ir
INFO: [Torch-TensorRT] - Module was provided as a torch.nn.Module, trying to script the module with torch.jit.script. In the event of a failure please preconvert your module to TorchScript
DEBUG: [Torch-TensorRT] - Settings requested for Lowering:
    torch_executed_modules: [
    ]
Traceback (most recent call last):
  File "/home/torch-tensorRT/test_torch-tensorrt.py", line 37, in <module>
    enabled_precisions={torch.half})
  File "/home/.local/lib/python3.6/site-packages/torch_tensorrt/_compile.py", line 97, in compile
    return torch_tensorrt.ts.compile(ts_mod, inputs=inputs, enabled_precisions=enabled_precisions, **kwargs)
  File "/home/.local/lib/python3.6/site-packages/torch_tensorrt/ts/_compiler.py", line 119, in compile
    compiled_cpp_mod = _C.compile_graph(module._c, _parse_compile_spec(spec))
RuntimeError: outputs_[i]->uses().empty()INTERNAL ASSERT FAILED at "../torch/csrc/jit/ir/ir.cpp":1286, please report a bug to PyTorch. 

[github-actions[bot]]

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

[github-actions[bot]]

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

[narendasan]

@peri044 I tried running this model and seems like there is an off by one error in the _convolution converter. Can you take a look? I can share the model with you over slack

[github-actions[bot]]

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days