Cant import torch_tensorrt

[sanath-tech]

ERROR: from torch.fx.passes.pass_manager import PassManager

ModuleNotFoundError: No module named 'torch.fx.passes.pass_manager'

• PyTorch Version : 1.11
• CPU Architecture: jetson AGX xavier
• OS (e.g., Linux):
• How you installed PyTorch: nvidia forum wheel
• Build command you used (if compiling from source):
• Are you using local sources or building from archives:
• Python version:3.8
• CUDA version: 11.4

[yinghai]

@frank-wei could take a look? It seems quite unexpected.

[frank-wei]

@sanath-tech
from torch.fx.passes.pass_manager import PassManager is not in pytorch 1.11. It appears in pytorch 1.12. FX path is new to Torch-TensorRT and does not have official wheel or release branch yet. It is suggested to be used with pytorch 1.12 and later on master branch. We will release the newest Torch-TensorRT branch in the next 2 weeks which works with pytorch 1.12. Please stay tuned.

[sanath-tech]

Is FX path is not in Torch-Tensorrt ,then why is it asking for it when I "import torch_tensorrt"

[frank-wei]

Is FX path is not in Torch-Tensorrt ,then why is it asking for it when I "import torch_tensorrt"

It is in Torch-TensorRT if you build from master. https://github.com/pytorch/TensorRT/blob/e113b48006b4a944859e206d4bd2337242b0af61/py/torch_tensorrt/init.py#L94

[sanath-tech]

Yes , i built from source using bazel as mentioned in Torch website for jetson

[sanath-tech]

still i couldnt solve the problem i tried both with torch 1.11 and torch 1.12

[sph1n3x]

@sanath-tech from torch.fx.passes.pass_manager import PassManager is not in pytorch 1.11. It appears in pytorch 1.12. FX path is new to Torch-TensorRT and does not have official wheel or release branch yet. It is suggested to be used with pytorch 1.12 and later on master branch. We will release the newest Torch-TensorRT branch in the next 2 weeks which works with pytorch 1.12. Please stay tuned.

Any chance of CUDA 11.6 and TensorRT 8.4.x support in the next release? Would be interesting to know!

[sanath-tech]

I successfully installed Torch-TensorRT in AGX Xavier, but now when i run below mentioned code, which is example code in torch website, i get below errors

import torch_tensorrt import torch import torchvision.models as models

model = models.mobilenet_v2(pretrained=True)

script_model = torch.jit.script(model)

spec = { "forward": torch_tensorrt.ts.TensorRTCompileSpec({ "inputs": [torch_tensorrt.Input([1, 3, 300, 300])], "enabled_precisions": {torch.float, torch.half}, "refit": False, "debug": False, "device": { "device_type": torch_tensorrt.DeviceType.GPU, "gpu_id": 0, "dla_core": 0, "allow_gpu_fallback": True }, "capability": torch_tensorrt.EngineCapability.default, "num_min_timing_iters": 2, "num_avg_timing_iters": 1, }) }

trt_model = torch._C._jit_to_backend("tensorrt", script_model, spec)

input = torch.randn((1, 3, 300, 300)).to("cuda").to(torch.half) print(trt_model.forward(input))

ERROR: Traceback (most recent call last): File "pretrained.py", line 20, in torch_tensorrt.ts.TensorRTCompileSpec({ File "/usr/local/lib/python3.8/dist-packages/torch_tensorrt/ts/_compile_spec.py", line 297, in TensorRTCompileSpec parsed_spec = _parse_compile_spec(compile_spec) File "/usr/local/lib/python3.8/dist-packages/torch_tensorrt/ts/_compile_spec.py", line 169, in _parse_compile_spec raise KeyError("Input specs should be either torch_tensorrt.Input or torch.Tensor, found types: {}".format( KeyError: "Input specs should be either torch_tensorrt.Input or torch.Tensor, found types: [<class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>]"

[narendasan]

I think you need to unpack the dict passed to torch_tensorrt.ts.TensorRTCompileSpec since it expect kwargs.

[DzAvril]

I successfully installed Torch-TensorRT in AGX Xavier, but now when i run below mentioned code, which is example code in torch website, i get below errors

import torch_tensorrt import torch import torchvision.models as models

model = models.mobilenet_v2(pretrained=True)

script_model = torch.jit.script(model)

spec = { "forward": torch_tensorrt.ts.TensorRTCompileSpec({ "inputs": [torch_tensorrt.Input([1, 3, 300, 300])], "enabled_precisions": {torch.float, torch.half}, "refit": False, "debug": False, "device": { "device_type": torch_tensorrt.DeviceType.GPU, "gpu_id": 0, "dla_core": 0, "allow_gpu_fallback": True }, "capability": torch_tensorrt.EngineCapability.default, "num_min_timing_iters": 2, "num_avg_timing_iters": 1, }) }

trt_model = torch._C._jit_to_backend("tensorrt", script_model, spec)

input = torch.randn((1, 3, 300, 300)).to("cuda").to(torch.half) print(trt_model.forward(input))

ERROR: Traceback (most recent call last): File "pretrained.py", line 20, in torch_tensorrt.ts.TensorRTCompileSpec({ File "/usr/local/lib/python3.8/dist-packages/torch_tensorrt/ts/_compile_spec.py", line 297, in TensorRTCompileSpec parsed_spec = _parse_compile_spec(compile_spec) File "/usr/local/lib/python3.8/dist-packages/torch_tensorrt/ts/_compile_spec.py", line 169, in _parse_compile_spec raise KeyError("Input specs should be either torch_tensorrt.Input or torch.Tensor, found types: {}".format( KeyError: "Input specs should be either torch_tensorrt.Input or torch.Tensor, found types: [<class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>]"

How do you fix the error "ModuleNotFoundError: No module named 'torch.fx.passes.pass_manager'"?

[narendasan]

Newer PyTorch implements this API but this issue should be resolved in master regardless of 1.11 or 1.12. Reopen if the issue persists