❓ [Question] How to install "tensorrt" package?

[domef]

❓ Question

I'm trying to install torch-tensorrt on a Jetson AGX Xavier. I first installed pytorch 1.12.0 and torchvision 0.13.0 following this guide. Then I installed torch-tensorrt following this guide, and the compilation completed succesfully.

When I try to import torch_tensorrt it throws an error, saying it can't find a module named tensorrt. Where I can find this package?

Environment

I'm using a Jetson AGX Xavier with Jetpack 5.0.1.

[narendasan]

Are you able to import the tensorrt package explicitly on its own?

[domef]

I don't have tensorrt in my environment, in fact my question is: how can I install tensorrt?

[narendasan]

The tensorrt python package should come prepacked with Jetpack. Are you using the system python install or something like conda?

[domef]

I created a new environment with python3.8 (I installed torch-tensorrt in the virtual env).

[narendasan]

Can you try importing tensorrt with the system python?

[domef]

I will try, but how can I install tensorrt on another virtual environment?

[narendasan]

I am not sure, TensorRT gets install via Jetpack (a debian package internally). You could try to extract the wheel to install. Alternatively you can check if it is distributed for aarch64-jetson on nvidia-pyindex

[domef]

I tried to import tensorrt with the system python and it is imported successfully. To install tensorrt in my environment I tried this:

pip install nvidia-pyindex

and then:

pip install nvidia-tensorrt

but I get this error with the last command:

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting nvidia-tensorrt
  Downloading nvidia-tensorrt-0.0.1.dev5.tar.gz (7.9 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-buu37o4p/nvidia-tensorrt_25bac5152bd94d91ab62ef9f8c6e8ea0/setup.py", line 150, in <module>
          raise RuntimeError(open("ERROR.txt", "r").read())
      RuntimeError:
      ###########################################################################################
      The package you are trying to install is only a placeholder project on PyPI.org repository.
      This package is hosted on NVIDIA Python Package Index.
      
      This package can be installed as:
      ```
      $ pip install nvidia-pyindex
      $ pip install nvidia-tensorrt
      ```
      ###########################################################################################
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[narendasan]

Sorry this is so convoluted.

This error is likely due to NVIDIA not shipping aarch64 tensorrt packages on the nvidia-pyindex, pip falls back to a dummy package on the main pypi index which points you to the nvidia-index (confusingly for Jetson users). Not sure what the next step is here. You can look on the host machine you used to flash the jetson for inside the folder which contains all the resources to build the jetpack image for the debian package for tensorrt. Unpacking that should have the whl file in it.

[msaroufim]

@narendasan I'm getting the same error on stock Ubuntu - any suggestions for next steps? I litterally followed these steps from your release notes

pip install nvidia-pyindex
pip install nvidia-tensorrt==8.4.3.1
pip install torch-tensorrt==1.2.0 -f https://github.com/pytorch/tensorrt/releases

And I get this error

(tensor) ubuntu@ip-172-31-17-70:~/torch2trt$ pip install torch-tensorrt==1.2.0 -f https://github.com/pytorch/tensorrt/releases
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Looking in links: https://github.com/pytorch/tensorrt/releases
ERROR: Could not find a version that satisfies the requirement torch-tensorrt==1.2.0 (from versions: 0.0.0, 0.0.0.post1)
ERROR: No matching distribution found for torch-tensorrt==1.2.0

And my OS version

(tensor) ubuntu@ip-172-31-17-70:~/torch2trt$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.5 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.5 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

[gaurav274]

This fixed it for me: https://github.com/pytorch/TensorRT/issues/1371#issuecomment-1256035010