NVIDIA
poetry install failure of TensorRT 8.6.1 when installing
Description
I prefer poetry for managing dependencies, but tensorrt fails to install due to lack of PEP-517 support.
poetry add tensorrt
$ poetry add tensorrt
Using version ^8.6.1 for tensorrt
Updating dependencies
Resolving dependencies... (0.7s)
Writing lock file
Package operations: 1 install, 0 updates, 0 removals
• Installing tensorrt (8.6.1): Failed
ChefBuildError
Backend subprocess exited when trying to invoke build_wheel
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/tensorrt
copying tensorrt/__init__.py -> build/lib/tensorrt
running egg_info
writing tensorrt.egg-info/PKG-INFO
writing dependency_links to tensorrt.egg-info/dependency_links.txt
writing requirements to tensorrt.egg-info/requires.txt
writing top-level names to tensorrt.egg-info/top_level.txt
reading manifest file 'tensorrt.egg-info/SOURCES.txt'
adding license file 'LICENSE.txt'
writing manifest file 'tensorrt.egg-info/SOURCES.txt'
installing to build/bdist.linux-x86_64/wheel
running install
/tmp/tmpx8nxodew/.venv/bin/python: No module named pip
Traceback (most recent call last):
File "/home/ruckc/.local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/ruckc/.local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/ruckc/.local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 416, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
self.run_setup()
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 487, in run_setup
super(_BuildMetaLegacyBackend,
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup
exec(code, locals())
File "<string>", line 49, in <module>
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup
return distutils.core.setup(**attrs)
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 378, in run
self.run_command("install")
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/tmp/tmpx8nxodew/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "<string>", line 43, in run
File "<string>", line 41, in install_dep
File "/usr/lib/python3.10/subprocess.py", line 456, in check_returncode
raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '['/tmp/tmpx8nxodew/.venv/bin/python', '-m', 'pip', 'install', 'tensorrt_libs==8.6.1', '--index-url', 'https://pypi.nvidia.com']' returned non-zero exit status 1.
at ~/.local/lib/python3.10/site-packages/poetry/installation/chef.py:152 in _prepare
148│
149│ error = ChefBuildError("\n\n".join(message_parts))
150│
151│ if error is not None:
→ 152│ raise error from None
153│
154│ return path
155│
156│ def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:
pip install --use-pep517 tensorrt
$ pip install --use-pep517 tensorrt
Collecting tensorrt
Downloading tensorrt-8.6.1.tar.gz (16 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: tensorrt
Building wheel for tensorrt (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for tensorrt (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [59 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/tensorrt
copying tensorrt/__init__.py -> build/lib/tensorrt
running egg_info
writing tensorrt.egg-info/PKG-INFO
writing dependency_links to tensorrt.egg-info/dependency_links.txt
writing requirements to tensorrt.egg-info/requires.txt
writing top-level names to tensorrt.egg-info/top_level.txt
reading manifest file 'tensorrt.egg-info/SOURCES.txt'
adding license file 'LICENSE.txt'
writing manifest file 'tensorrt.egg-info/SOURCES.txt'
installing to build/bdist.linux-x86_64/wheel
running install
/home/ruckc/.cache/pypoetry/virtualenvs/embeddings-qEYH5wU2-py3.10/bin/python: No module named pip
Traceback (most recent call last):
File "/home/ruckc/.cache/pypoetry/virtualenvs/embeddings-qEYH5wU2-py3.10/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/ruckc/.cache/pypoetry/virtualenvs/embeddings-qEYH5wU2-py3.10/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/ruckc/.cache/pypoetry/virtualenvs/embeddings-qEYH5wU2-py3.10/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 416, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
self.run_setup()
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 487, in run_setup
super(_BuildMetaLegacyBackend,
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup
exec(code, locals())
File "<string>", line 49, in <module>
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup
return distutils.core.setup(**attrs)
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 378, in run
self.run_command("install")
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/tmp/pip-build-env-95qyxdbs/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "<string>", line 43, in run
File "<string>", line 41, in install_dep
File "/usr/lib/python3.10/subprocess.py", line 456, in check_returncode
raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '['/home/ruckc/.cache/pypoetry/virtualenvs/embeddings-qEYH5wU2-py3.10/bin/python', '-m', 'pip', 'install', 'tensorrt_libs==8.6.1', '--index-url', 'https://pypi.nvidia.com']' returned non-zero exit status 1.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for tensorrt
Failed to build tensorrt
ERROR: Could not build wheels for tensorrt, which is required to install pyproject.toml-based projects
Environment
Ubuntu 22.04 & Ubuntu 23.04
TensorRT Version: 8.6.1
NVIDIA GPU: 2080ti
NVIDIA Driver Version: 535.98
CUDA Version: 12.2
CUDNN Version: 8.9.2.26-1+cuda12.1
Operating System:
Python Version (if applicable): 3.10
Tensorflow Version (if applicable): 2.12.0
Baremetal or Container (if so, version): both
Steps To Reproduce
Commands or scripts:
pip install --use-pep517 tensorrt
Have you tried the latest release?: yes
I don't know much about it, @pranavm-nvidia could you please kindly help :-)
Encountered this today and fighting to get around it.
Obviously, adding tensorrt to poetry.tool.dependencies is the attempted way to do this, but that doesn't work because… tensorrt's setup tries to run
pip install tensorrt_libs==8.6.1 --index-url https://pypi.nvidia.com
pip install tensorrt_bindings==8.6.1 --index-url https://pypi.nvidia.com
as a part of its installation process:
import sys
from setuptools import setup
from setuptools.command.install import install
import subprocess as sp
tensorrt_module = "tensorrt"
class InstallCommand(install):
def run(self):
def install_dep(package_name):
status = sp.run(
[
sys.executable,
"-m",
"pip",
"install",
"{:}==8.6.1".format(package_name),
"--index-url",
"https://pypi.nvidia.com",
]
)
status.check_returncode()
install_dep("{:}_libs".format(tensorrt_module))
install_dep("{:}_bindings".format(tensorrt_module))
install.run(self)
setup(
name=tensorrt_module,
version="8.6.1",
description="A high performance deep learning inference library",
long_description="A high performance deep learning inference library",
author="NVIDIA Corporation",
license="Proprietary",
classifiers=[
"License :: Other/Proprietary License",
"Intended Audience :: Developers",
"Programming Language :: Python :: 3",
],
packages=[tensorrt_module],
extras_require={"numpy": "numpy"},
package_data={tensorrt_module: ["*.so*", "*.pyd", "*.pdb"]},
include_package_data=True,
zip_safe=True,
keywords="nvidia tensorrt deeplearning inference",
url="https://developer.nvidia.com/tensorrt",
download_url="https://github.com/nvidia/tensorrt/tags",
cmdclass={"install": InstallCommand},
)
So instead I tried adding those dependencies directly, but it's really not going to work well because then the tensorrt package itself that essentially just reexports tensorrt_bindings won't be available:
[[tool.poetry.source]]
name = "nvidia"
url = "https://pypi.nvidia.com"
priority = "supplemental"
[tool.poetry.dependencies.tensorrt_libs]
version ="*"
markers = "sys_platform == 'linux'"
source = "nvidia"
[tool.poetry.dependencies.tensorrt_binding]
version ="*"
markers = "sys_platform == 'linux'"
source = "nvidia"
No dice:
$ poetry update
Warning: In a future version of Poetry, PyPI will be disabled automatically if at least one custom source is configured with another priority than 'explicit'. In order to avoid a breaking change and make your pyproject.toml forward compatible, add PyPI explicitly via 'poetry source add pypi'. By the way, this has the advantage that you can set the priority of PyPI as with any other source.
Updating dependencies
Resolving dependencies... (4.0s)Source (nvidia): Authorization error accessing https://pypi.nvidia.com/tensorrt-binding/
Resolving dependencies... (16.1s)Source (nvidia): Authorization error accessing https://pypi.nvidia.com/tensorrt-binding/
Resolving dependencies... (21.4s)Source (nvidia): Authorization error accessing https://pypi.nvidia.com/tensorrt-binding/
Resolving dependencies... (27.4s)
I'm going to end up having to effectively sideload it, or give up on installing tensorrt through the Python ecosystem and use the DEB package instead for my environments.
Hi all, is it possible that you can provide a minimal reproduce steps so that I can reproduce and file internal bug.
Hi @ruckc,
I was able to reproduce this issue by installing Poetry in an Ubuntu 20.04 container and creating a simple pyproject.toml file, such as the following:
[tool.poetry]
name = "test"
version = "0.0.1"
authors = ["User <[email protected]>"]
description = "Test project"
[tool.poetry.dependencies]
python = "^3.8"
I found a workaround, which isn't nice, but at least makes the Poetry environment functional and the tensorrt module can be imported. We are not following PEP-517 yet as you mentioned, which would help with this situation.
I first set the environment variable NVIDIA_TENSORRT_DISABLE_INTERNAL_PIP=True and then used the following commands to add the dependency.
poetry source add --priority=supplemental nvidia https://pypi.nvidia.com/
poetry add --source nvidia tensorrt tensorrt-libs tensorrt-bindings
Hi @ruckc,
I was able to reproduce this issue by installing Poetry in an Ubuntu 20.04 container and creating a simple
pyproject.tomlfile, such as the following:[tool.poetry] name = "test" version = "0.0.1" authors = ["User <[email protected]>"] description = "Test project" [tool.poetry.dependencies] python = "^3.8"I found a workaround, which isn't nice, but at least makes the Poetry environment functional and the
tensorrtmodule can be imported. We are not following PEP-517 yet as you mentioned, which would help with this situation.I first set the environment variable
NVIDIA_TENSORRT_DISABLE_INTERNAL_PIP=Trueand then used the following commands to add the dependency.poetry source add --priority=supplemental nvidia https://pypi.nvidia.com/ poetry add --source nvidia tensorrt tensorrt-libs tensorrt-bindings
Doing the above still installs tensorrt-bindings (8.6.1) and tensorrt-libs (8.6.1) instead of the latest and currently required packages. How does one solve this? Do I explicitly mention the versions needed in the poetry config? Is this not a bad practice? @zeroepoch
@Bhuvanesh09,
Doing the above still installs tensorrt-bindings (8.6.1) and tensorrt-libs (8.6.1) instead of the latest and currently required packages. How does one solve this?
The last command now becomes:
poetry add --source nvidia tensorrt-cu12 tensorrt-cu12-libs tensorrt-cu12-bindings
The real packages are named according to their CUDA major version starting with TensorRT 10.0.
@zeroepoch Thanks for the prompt reply,
Unfortunately, I am wanting tensorrt_llm compatible version which currently is 9.3.0.post12.dev1. Even though tensorrt itself is correctly installed, the tensorrt-bindings and tensorrt-libs versions installed are 8.6.1. Currently, I am able to make it work by hardcoding the versions for the above packages in my pyproject.toml. Is there any workaround where I don't have to do this?
Thanks!
@Bhuvanesh09,
TRT 9.3.0 is a pre-release version so you'll need to configure Poetry to use pre-release versions. I haven't researched this myself. The other problem I found from experimenting earlier is that Poetry is not "smart" enough to figure out the dependencies here. Since the libs and bindings packages are installed during the installation of the tensorrt frontend package it doesn't see them as real dependencies. In my experiments if I told Poetry to install tensorrt it would add tensorrt-cu12 then stop there unless I added all 3 packages manually. If you want to avoid specifying the other packages then you'll need to use a requirements.txt file I think so pip handles the installation instead of Poetry.
Also, I'm not sure how you expect Poetry to install a specific version, but then not want to specify that version. I'm a bit confused because that would be like asking to install numpy 1.23.0 when the latest version might be 1.26.0 and using pip install numpy to do so. If you're expecting the frontend package to depend on the right libs and bindings packages then it won't work with Poetry unless you come up with some clever hack I'm unaware of or pin all subpackages as well as you're suggesting.
The other problem I found from experimenting earlier is that Poetry is not "smart" enough to figure out the dependencies here. Since the libs and bindings packages are installed during the installation of the tensorrt frontend package it doesn't see them as real dependencies.
This is exactly the issue that I am trying to solve. My intention is to use tensorrt_llm in a package that I am building without much hassle.
Also, I'm not sure how you expect Poetry to install a specific version, but then not want to specify that version. I'm a bit confused because that would be like asking to install numpy 1.23.0 when the latest version might be 1.26.0 and using pip install numpy to do so.
The issue boils down to the dependency resolver not working as it is supposed to do. For example, installing pandas would automatically install the best version of numpy which is compatible even if a newer version of numpy is available. This same is observed when one installs tensorrt_llm using pip. The most appropriate versions of tensorrt, tensorrt-libs, and tensorrt-bindings are automatically resolved. I presume that all of this is because of use of pip within pip in the installation process of tensorrt.
Anyhow, the way I am going ahead with building my package is:
- Check which version of tensorrt is compatible with tensorrt_llm by installing it in an isolated environment and doing pip freeze.
- Manually force poetry to install that version for my package by mentioning the version in pyproject.toml
I feel that this method should work since I have little reason to automatically upgrade tensorrt_llm as a dependency to my package. As long as I want to use v0.9.0 of tensorrt_llm, there shouldn't be any issues.
Thanks for the discussion and comments @zeroepoch . I have a better clarity, and confidence in the process.