TensorRT Can please somebody give a clear explanation of how to install torch-tensorrt on Windows?

❓ Question

Hello,

I've encountered problems installing torch-tensorrt on Windows 10

No matter how I try, how many sources I look up to, there is no clear explanation on how to do everything. The documentation is vague, and because I am used to working with python code, which does everything for you, that is pip install... python code.py, and nothing more is required, I do not have as much experience with cmake, building libraries, files, and c++, which makes it very difficult to follow along the installation process.

Now I've tried to follow along instructions from the main page

pip install torch-tensorrt doesn't work downloaded zip file of this repository; python setup.py install also doesn't work

installed bazel modified the workspace, still nothing

tried to directly import into code py/torch-tensorrt - nothing

then inside the py folder opened command prompt ant typed in:

bazel build //:libtorchtrt --compilation_mode=dbg

and received this error:

`Starting local Bazel server and connecting to it... INFO: Repository libtorch instantiated at: D:/pyth/tensorrt-main/WORKSPACE:53:13: in Repository rule http_archive defined at: C:/users/tomas/_bazel_tomas/r4zfvyvs/external/bazel_tools/tools/build_defs/repo/http.bzl:372:31: in WARNING: Download from https://download.pytorch.org/libtorch/nightly/cu121/libtorch-cxx11-abi-shared-with-deps-latest.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.ContentLengthMismatchException Bytes read 2210658461 but wanted 2501377827 ERROR: An error occurred during the fetch of repository 'libtorch': Traceback (most recent call last): File "C:/users/tomas/_bazel_tomas/r4zfvyvs/external/bazel_tools/tools/build_defs/repo/http.bzl", line 132, column 45, in _http_archive_impl download_info = ctx.download_and_extract( Error in download_and_extract: java.io.IOException: Error downloading [https://download.pytorch.org/libtorch/nightly/cu121/libtorch-cxx11-abi-shared-with-deps-latest.zip] to C:/users/tomas/_bazel_tomas/r4zfvyvs/external/libtorch/temp7217651597570855917/libtorch-cxx11-abi-shared-with-deps-latest.zip: Bytes read 2210658461 but wanted 2501377827 ERROR: D:/pyth/tensorrt-main/WORKSPACE:53:13: fetching http_archive rule //external:libtorch: Traceback (most recent call last): File "C:/users/tomas/_bazel_tomas/r4zfvyvs/external/bazel_tools/tools/build_defs/repo/http.bzl", line 132, column 45, in _http_archive_impl download_info = ctx.download_and_extract( Error in download_and_extract: java.io.IOException: Error downloading [https://download.pytorch.org/libtorch/nightly/cu121/libtorch-cxx11-abi-shared-with-deps-latest.zip] to C:/users/tomas/_bazel_tomas/r4zfvyvs/external/libtorch/temp7217651597570855917/libtorch-cxx11-abi-shared-with-deps-latest.zip: Bytes read 2210658461 but wanted 2501377827 ERROR: D:/pyth/tensorrt-main/core/util/logging/BUILD:13:11: //core/util/logging:logging depends on @libtorch//:libtorch in repository @libtorch which failed to fetch. no such package '@libtorch//': java.io.IOException: Error downloading [https://download.pytorch.org/libtorch/nightly/cu121/libtorch-cxx11-abi-shared-with-deps-latest.zip] to C:/users/tomas/_bazel_tomas/r4zfvyvs/external/libtorch/temp7217651597570855917/libtorch-cxx11-abi-shared-with-deps-latest.zip: Bytes read 2210658461 but wanted 2501377827 ERROR: Analysis of target '//:libtorchtrt' failed; build aborted: INFO: Elapsed time: 458.697s INFO: 0 processes. FAILED: Build did NOT complete successfully (64 packages loaded, 413 targets configured) Fetching https://download.pytorch.org/...orch-cxx11-abi-shared-with-deps-latest.zip; 2.1 GiB (2,210,121,825B) 446s

And also tried some other things, I cannot remember, but unsuccessfully.

THANK YOU FOR YOUR HELP IN ADVANCE

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

PyTorch Version: 2.1.2+cu121
OS : Windows 10
I am running python and pytorch straight from Windows, without any environment
Python version: 3.10.13
CUDA version: 12.1 update 1
GPU models and configuration: GTX 1660 TI

Jan 05 '24 02:01 ninono12345

Hello - we are working on a pip-installable Windows distribution for which a sample can be found on the windows_CI branch. This branch currently only supports our Dynamo paths (ir=dynamo, ir=torch_compile) and a limited TensorRT runtime (though #1943 will improve this runtime constraint). It does not require bazel to install.

In order to use it, the Torch distribution requires this patch: https://github.com/pytorch/pytorch/pull/111313, to enable Dynamo experimentally on Windows, since Torch does not yet fully support Dynamo on Windows. Then, once cloned, running python setup.py install from the root of the directory would begin the package installation. I have tested it successfully on Python 3.10.

Jan 05 '24 23:01 gs-olive

Hello again, thank you for your reply, @gs-olive, and I apologize, for replying so late. Thank you for pointing me to the windows_CI branch, I haven't noticed it then.

Now if I use git clone https://github.com/pytorch/TensorRT.git (even though this is the same link to non windows_CI, it's on windows_CI branch) and run python setup.py install I get Could not find bazel in PATH . This tells me that it is not the windows_CI being downloaded.

Now I downloaded the zip file format, I get the TensorRT windows_CI, when i run setup.py, i get an error, that this is not a git repository. I ran the command git init, that error is gone, but now I receive this error:

fatal: Needed a single revision Traceback (most recent call last): File "D:\pyth\TensorRT-windows_CI\setup.py", line 104, in <module> __version__ = f"{get_base_version()}.dev0+{get_git_revision_short_hash()}" File "D:\pyth\TensorRT-windows_CI\setup.py", line 43, in get_git_revision_short_hash subprocess.check_output(["git", "rev-parse", "--short", "HEAD"]) File "C:\Users\Tomas\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 421, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "C:\Users\Tomas\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['git', 'rev-parse', '--short', 'HEAD']' returned non-zero exit status 128

specifically the line fatal: Needed a single revision I do not understand, and once again, can't find any help online...

Thank you for your help

Jan 10 '24 20:01 ninono12345

Hello - thanks for the follow-up. To clone a specific branch from the command line, I use:

git clone --single-branch --branch windows_CI https://github.com/pytorch/TensorRT.git

This should resolve the git issues related to that repository. Then, python setup.py install should avoid bazel.

Jan 12 '24 22:01 gs-olive

Hi again @gs-olive, I've successfully installed it like you said, but now when i try to import torch_tensorrt, I get this error:

C:\Users\Tomas\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt_Device.py:19: UserWarning: Unable to import torchscript frontend core and torch-tensorrt runtime. Some dependent features may be unavailable. warnings.warn( Traceback (most recent call last): File "D:\pyth\pytracking-master\band5.py", line 22, in import torch_tensorrt File "C:\Users\Tomas\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt_init_.py", line 84, in from torch_tensorrt._compile import * # noqa: F403 File "C:\Users\Tomas\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt_compile.py", line 24, in from torch_tensorrt.dynamo.compiler import compile as dynamo_compile File "C:\Users\Tomas\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt\dynamo_init.py", line 10, in from .compiler import compile File "C:\Users\Tomas\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt\dynamo_compiler.py", line 45, in from torch_tensorrt.dynamo.lowering import apply_lowering_passes, get_decompositions File "C:\Users\Tomas\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt\dynamo\lowering_init.py", line 7, in from .repair_input_aliasing import repair_input_aliasing File "C:\Users\Tomas\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt\dynamo\lowering_repair_input_aliasing.py", line 4, in from torch_tensorrt.dynamo.lowering.passes.pass_utils import get_tensor_placeholders File "C:\Users\Tomas\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt\dynamo\lowering\passes_init.py", line 1, in from ._aten_lowering_pass import * File "C:\Users\Tomas\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt\dynamo\lowering\passes_aten_lowering_pass.py", line 6, in from .constant_folding import constant_fold File "C:\Users\Tomas\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt\dynamo\lowering\passes\constant_folding.py", line 16, in from torch._inductor.constant_folding import ( ModuleNotFoundError: No module named 'torch._inductor.constant_folding'

Jan 12 '24 22:01 ninono12345

Thanks for testing that out - I have seen this error before; it relates to the torch version being used. The branch is based on the latest Torch nightly, which has the torch._inductor.constant_folding module, but Torch 2.1.2 does not have that module. I just updated the branch to fix that issue, so windows_CI should get past that issue now if you pull the latest changes and reinstall.

Jan 13 '24 07:01 gs-olive

Thank you @gs-olive for your fast reply. Now I'm really confused. I installed torch_tensorrt, now when I try to import torch_tensorrt I get this error and the program stops: C:\Users\Tomas\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt\_Device.py:19: UserWarning: Unable to import torchscript frontend core and torch-tensorrt runtime. Some dependent features may be unavailable. warnings.warn(

for some reason at first when I installed it I was having some problems, so I did pip uninstall torch_tensorrt, and then ran again python setup.py install, and I get the error

Or for it to work ok, I need to download the pytorch 2.2.0 nightly?

Jan 13 '24 14:01 ninono12345

@gs-olive Hello, I cloned it again as well and it has the same error with "returned non-zero exit status 128" in the end. Can we have some kind of a step by step guide, what exactly we have to do?

1.) Download this that and that. 2.) Add this that and that to the PATH 3.) Then install Python 3.10 4.) Then run setup.py

Something like this I mean.

Thanks a lot.

Jan 13 '24 20:01 jensdraht1999

I still get the following error using the latest commit (https://github.com/pytorch/TensorRT/commit/3c099ef24f8663e578a61bd871a290368e8f37aa at the time of writing) from the windows_CI branch when trying to import torch_tensorrt.

Warning (from warnings module):
  File "C:\Python311\Lib\site-packages\torch_tensorrt\_Device.py", line 19
    warnings.warn(
UserWarning: Unable to import torchscript frontend core and torch-tensorrt runtime. Some dependent features may be unavailable.
Traceback (most recent call last):
  File "<pyshell#2>", line 1, in <module>
    import torch_tensorrt
  File "C:\Python311\Lib\site-packages\torch_tensorrt\__init__.py", line 90, in <module>
    from torch_tensorrt.logging import *
  File "C:\Python311\Lib\site-packages\torch_tensorrt\logging.py", line 4, in <module>
    from torch_tensorrt._C import (
ModuleNotFoundError: No module named 'torch_tensorrt._C'

Jan 14 '24 10:01 HolyWu

@HolyWu I don't know exactly why the problem of you is happening, because I'm not the dev of this, but perhaps you could try Python 3.10 because the dev said it.

Jan 14 '24 18:01 jensdraht1999

@gs-olive Hello, I cloned it again as well and it has the same error with "returned non-zero exit status 128" in the end. Can we have some kind of a step by step guide, what exactly we have to do?

1.) Download this that and that. 2.) Add this that and that to the PATH 3.) Then install Python 3.10 4.) Then run setup.py

Something like this I mean.

Thanks a lot.

By the way I have solved it myself. Here is how:

1.) Before all of this, please install Python 3.11. 2.) And then do this: "pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121" 3.) Download Cuda 12.1 (only this) 4.) Download Cudnn 5.) Download Nvida Tensorrt 6.) Install Cuda 7.) Copy the cudnn content(whats in it) to the folder C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1. 8.) Put Nvidia Tensorrt to the Enviroment Path. 9.) Clone the Pytorch/Tensorrt via this command ONLY (nothing else does not work): git clone --single-branch --branch windows_CI https://github.com/pytorch/TensorRT.git 10.) Then you have to do: "cd folderwhereithasbeencopied" 11.) Then you have to do: "python setup.py"

Setup should install the library.

However I have not tried more than this for now. Some kind of example would be helpful?

Jan 16 '24 10:01 jensdraht1999

Hello - thanks for the comments on this thread. To address them:

@ninono12345 - the UserWarning: Unable to import torchscript frontend core and torch-tensorrt runtime. is expected currently, since the version of Torch-TensorRT in the windows_CI branch does not support TorchScript; it only supports the ir="dynamo" and ir="torch_compile" IRs when using this patch: https://github.com/pytorch/pytorch/pull/111313.

@HolyWu - thanks for this catch/report - torch_tensorrt._C is not installed in the Windows build on windows_CI, so that import should not be present in this experimental branch. I've added a commit to remove it.

@jensdraht1999 - the steps provided are very close to what I am using on this experimental branch to get a working build. For a detailed set of steps I am currently using with this experimental branch, here is what I have:

Steps for Dynamo-Only, Python-Runtime-Only Windows build using `windows_CI` Branch

Ensure CUDA is Installed (preferably 12.x). See official CUDA installation steps for Windows if not.
Install your preferred TensorRT Python version. Currently, I use pip install --no-cache-dir --extra-index-url https://pypi.nvidia.com tensorrt==9.2.0.post12.dev5. You can also install other TensorRT versions with the Python bindings using the official TensorRT Windows installation docs, which currently do not use pip.
Install the nightly distribution of Torch, for instance: pip install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu121
Apply this patch: https://github.com/pytorch/pytorch/pull/111313. Since Torch does not officially support Windows for Dynamo, but most of the required functionality is there, we use this patch as an experimental workaround while the pending issues for torch.compile on Windows are addressed. I currently apply the patch by modifying the torch/_dynamo/eval_frame.py file, for which the location can be found by running pip show torch.
Clone the branch: git clone --single-branch --branch windows_CI https://github.com/pytorch/TensorRT.git
cd into the directory containing the cloned repo. For instance, cd TensorRT
Run python setup.py install to install torch_tensorrt
Open a Python interpreter and run import torch; import torch_tensorrt. There may be warnings about TorchScript and Torchvision, but the import should succeed.
Try an example run, for instance python examples/dynamo/torch_compile_resnet_example.py, which runs ResNet using this tutorial.

Note: Windows support for the Dynamo paths is in an experimental state, and there are no guarantees on functionality + it may not work in some compilation cases since it is under active development. Note: To run torch_tensorrt.compile on Windows using the windows_CI branch, it is required to use ir="dynamo" or ir="torch_compile" and use_python_runtime=True.

Jan 16 '24 20:01 gs-olive

@gs-olive I have done all of this. But Step 8 is a problem. Step 9 doesn't really work I think. Please look at the pictures. Would you recommend me to use Python 3.10? By the way I have totally reinstalled Python 3.11 deleted all the sitepackage folder and more to have it all clean and also cuda 12.1.

Step 8:

Step 9: If I run the example file without changing anything:

log.txt

What could be the problem?

Jan 16 '24 22:01 jensdraht1999

Hi - thanks for the follow-up. I do not think Python 3.10 vs 3.11 should make a difference here, though I am using Python 3.10. Regarding the attached image, in the IDE on the left, it seems that there is an error finding a cudnn file, whereas in the IDE on the right, there is just the expected warning. Did anything change between the two runs to make the import succeed, as it does on the left?

Regarding the error logs, that is due to the version of the example which I had linked in the original message. I have fixed the link now, and it uses the correct configuration parameters for Windows. It is linked here as well for convenience.

Jan 16 '24 23:01 gs-olive

@gs-olive When I "import torch; import torch_tensorrt" in IDLE, then it will throw multiple .dll "procedure entry point not found" (german "prozedureinsprungpunkt nicht gefunden") errors. After I click them away, then it will run the right one.

The files are actually there, where they are suppossed to be. Please look:

Also the eval file is there where it belongs:

This is the log for example file after that you have linked right now:

d.txt

Jan 16 '24 23:01 jensdraht1999

@gs-olive I tried Python 3.10 as well it did not work. Could you upload your site-packages folder to the internet so I could try it out?

Jan 17 '24 21:01 jensdraht1999

Got it - thanks for sharing this. It seems that the issue there is in Dynamo and not Torch-TensorRT. If the below snippet also fails on your machine, that would indicate a Torch issue.

import torch
import torchvision.models as models


model = models.resnet18(pretrained=True).half().eval().to("cuda")
inputs = [torch.randn((1, 3, 224, 224)).to("cuda").half()]
optimized_model = torch.compile(model, backend="eager")
optimized_model (*inputs)

Additionally, I am not using IDLE as my Python interpreter - I have installed Python for Windows from the Python site directly and I am invoking it in Powershell - I think this could also potentially affect Torch. The versions of the relevant packages on my machine are:

$ python -m pip list
Package                  Version
------------------------ ------------------------ 
tensorrt                 9.2.0.post12.dev5
tensorrt-bindings        9.2.0.post12.dev5
tensorrt-libs            9.2.0.post12.dev5
torch                    2.3.0.dev20240116+cu121

Jan 18 '24 03:01 gs-olive

The example you have provided did give me the same error, so my Pytorch Nightly does somehow not work. So I have to work it out.

My depedencies:

C:\Users\King>python -m pip list Package Version

black 23.12.1 certifi 2023.11.17 cfgv 3.4.0 charset-normalizer 3.3.2 clang-format 14.0.6 click 8.1.7 colorama 0.4.6 distlib 0.3.8 exceptiongroup 1.2.0 expecttest 0.2.1 filelock 3.9.0 fsspec 2023.4.0 huggingface-hub 0.20.2 identify 2.5.33 idna 3.6 iniconfig 2.0.0 isort 5.13.2 Jinja2 3.1.2 MarkupSafe 2.1.3 mpmath 1.2.1 mypy 1.8.0 mypy-extensions 1.0.0 networkx 3.0rc1 nodeenv 1.8.0 numpy 1.26.3 nvidia-cublas-cu12 12.3.4.1 nvidia-cuda-nvrtc-cu12 12.3.107 nvidia-cuda-runtime-cu12 12.3.101 nvidia-cudnn-cu12 8.9.7.29 packaging 23.2 parameterized 0.9.0 pathspec 0.12.1 pillow 10.2.0 pip 23.3.2 platformdirs 4.1.0 pluggy 1.3.0 pre-commit 3.6.0 pytest 7.4.4 PyYAML 6.0.1 regex 2023.12.25 requests 2.31.0 ruff 0.1.13 safetensors 0.4.1 setuptools 69.0.3 sympy 1.11.1 tensorrt 9.2.0.post12.dev5 tensorrt-bindings 9.2.0.post12.dev5 tensorrt-libs 9.2.0.post12.dev5 timm 0.9.12 tokenizers 0.15.0 tomli 2.0.1 torch 2.3.0.dev20240117+cu121 torch_tensorrt 2.2.0.dev0+2c304e4b torchvision 0.16.2 tqdm 4.66.1 transformers 4.36.2 typing_extensions 4.8.0 urllib3 2.1.0 virtualenv 20.25.0 wheel 0.42.0

Jan 18 '24 18:01 jensdraht1999

@gs-olive Just one question. I think I have ruled out the problem. I think Torchvision is definitly needed and also installed the nightly version, but this is making problem.

Can i just know from you, which torchvision and torchaudio you have installed. I think this would make this problem go away.

Jan 18 '24 20:01 jensdraht1999

Okay fixed the problem with Torchvision like this. Please try to download them like this:

pip install --pre torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu121

While downloading it, you will the see the download link for .whl files. Put the link in a browser, download them. After that put them in a folder in the desktop for example and install via this command:

pip install "C:\Users{UserName}\Desktop\whl files\torchaudio-2.2.0.dev20240118+cu121-cp310-cp310-win_amd64.whl" --force-reinstall --no-deps

pip install "C:\Users{UserName}\Desktop\whl files\torchvision-0.18.0.dev20240118+cu121-cp310-cp310-win_amd64.whl" --force-reinstall --no-deps

This will install those both packages regardless of their dependencies.

However the problem on the original example exists, but I will look at it now.

Jan 18 '24 20:01 jensdraht1999

Thanks for the update. I do not have torchaudio installed, and my versions of torch and torchvision are:

torch                    2.3.0.dev20240116+cu121
torchvision              0.18.0.dev20240116+cu121

I installed these using something very similar to what you have written:

pip uninstall -y torch torchvision
pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu121

Jan 18 '24 22:01 gs-olive

@gs-olive Okay now I have fixed it, the rest of the errors are the expected ones. The problem was this:

You have to delete following folder:

"C:\Users{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\nvidia\cudnn"

Because it seems that torch already has those files somehow and now the dll errors disappeard.

Jan 19 '24 20:01 jensdraht1999

Tips: Do NOT USE IDLE. There is really something broke with it. Use Cmd or Powershell or something else.

It really worked with the example, if run through cmd.

There's still an error, but it is definitly loading everything into VRAM.

This error here is excepted, I mean in the end of the script ? @gs-olive

Thanks again for your help and patience. @gs-olive

log.log

Jan 20 '24 04:01 jensdraht1999

Updated with 12.3 cuda and made all the steps again. Now it seems to work better, but I'm not sure, if the example provided is suppossed to have exactly this error here below. And also just deleted the .dll files in the folder "C:\Users{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\nvidia\cudnn", everything else stayed in there. The error seems to be less problematic. And also run in CMD, NOT in IDLE.

log.log

Jan 20 '24 22:01 jensdraht1999

Thanks for the details - the error at the end is not expected and I want to determine whether the issue is in torch.compile or tensorrt. Does an error appear when compiling the Torch model from this comment: https://github.com/pytorch/TensorRT/issues/2577#issuecomment-1897730589?

Jan 24 '24 05:01 gs-olive

@gs-olive I do it like this:

The code I have saved in a file called 1.py. The content:

import torch
import torchvision
model = models.resnet18(pretrained=True).half().eval().to("cuda")
inputs = [torch.randn((1, 3, 224, 224)).to("cuda").half()]
optimized_model = torch_tensorrt.compile(model, backend="eager")
optimized_model (*inputs)

I run via CMD. This happens:

C:\Users\{UserName}>C:\Users\{UserName}\Desktop\1.py
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\utils\generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  _torch_pytree._register_pytree_node(
Traceback (most recent call last):
  File "C:\Users\{UserName}\Desktop\1.py", line 5, in <module>
    model = models.resnet18(pretrained=True).half().eval().to("cuda")
NameError: name 'models' is not defined

C:\Users\{UserName}>

But if I run this code here via CMD:

import torch
import torchvision.models as models
model = models.resnet18(pretrained=True).half().eval().to("cuda")
inputs = [torch.randn((1, 3, 224, 224)).to("cuda").half()]
optimized_model = torch_tensorrt.compile(model, backend="eager")
optimized_model (*inputs)

This will happen:

Microsoft Windows [Version 10.0.19045.3930]
(c) Microsoft Corporation. Alle Rechte vorbehalten.

C:\Users\{UserName}>C:\Users\{UserName}\Desktop\whl\1.py
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\utils\generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  _torch_pytree._register_pytree_node(
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Traceback (most recent call last):
  File "C:\Users\{UserName}\Desktop\whl\1.py", line 6, in <module>
    optimized_model = torch_tensorrt.compile(model, backend="eager")
NameError: name 'torch_tensorrt' is not defined

C:\Users\{UserName}>

If I run this code via CMD:

import torch
import torchvision.models as models
import torch_tensorrt
model = models.resnet18(pretrained=True).half().eval().to("cuda")
inputs = [torch.randn((1, 3, 224, 224)).to("cuda").half()]
optimized_model = torch_tensorrt.compile(model, backend="eager")
optimized_model (*inputs)

This happens:

Microsoft Windows [Version 10.0.19045.3930]
(c) Microsoft Corporation. Alle Rechte vorbehalten.

C:\Users\{UserName}>C:\Users\{UserName}\Desktop\whl\1.py
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\utils\generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  _torch_pytree._register_pytree_node(
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt\_Device.py:19: UserWarning: Unable to import torchscript frontend core and torch-tensorrt runtime. Some dependent features may be unavailable.
  warnings.warn(
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
INFO:torch_tensorrt._compile:ir was set to default, using dynamo as ir
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\_dynamo\eval_frame.py:1329: UserWarning: export(f, *args, **kwargs) is deprecated, use export(f)(*args, **kwargs) instead.  If you don't migrate, we may break your export call in the future if your user defined kwargs conflict with future kwargs added to export(f).
  warnings.warn(
Traceback (most recent call last):
  File "C:\Users\{UserName}\Desktop\whl\1.py", line 7, in <module>
    optimized_model = torch_tensorrt.compile(model, backend="eager")
  File "C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt\_compile.py", line 226, in compile
    exp_program = torch_tensorrt.dynamo.trace(module, torchtrt_inputs, **kwargs)
  File "C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt\dynamo\_tracer.py", line 85, in trace
    exp_program = export(mod, tuple(torch_inputs), dynamic_shapes=dynamic_shapes)
  File "C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\export\__init__.py", line 193, in export
    return _export(
  File "C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\export\exported_program.py", line 79, in wrapper
    return fn(*args, **kwargs)
  File "C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\export\_trace.py", line 651, in _export
    gm_torch_level = _export_to_torch_ir(
  File "C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\export\_trace.py", line 273, in _export_to_torch_ir
    gm_torch_level, _ = torch._dynamo.export(
  File "C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\_dynamo\eval_frame.py", line 1334, in export
    return inner(*extra_args, **extra_kwargs)
  File "C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\_dynamo\eval_frame.py", line 1220, in inner
    result_traced = opt_f(*args, **kwargs)
  File "C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\_dynamo\eval_frame.py", line 410, in _fn
    return fn(*args, **kwargs)
  File "C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
TypeError: ResNet.forward() got an unexpected keyword argument 'disable_constraint_solver'

C:\Users\{UserName}>

I'm not sure, if something I have forgotten somehow. I have followed all your steps, step by step. Here are the pacakages:

The packages:
black                    23.12.1
certifi                  2022.12.7
cfgv                     3.4.0
charset-normalizer       2.1.1
clang-format             14.0.6
click                    8.1.7
colorama                 0.4.6
distlib                  0.3.8
exceptiongroup           1.2.0
expecttest               0.1.6
filelock                 3.13.1
fsspec                   2023.12.2
huggingface-hub          0.20.2
identify                 2.5.33
idna                     3.4
iniconfig                2.0.0
isort                    5.13.2
Jinja2                   3.1.2
MarkupSafe               2.1.3
mpmath                   1.2.1
mypy                     1.8.0
mypy-extensions          1.0.0
networkx                 3.0rc1
nodeenv                  1.8.0
numpy                    1.24.1
nvidia-cublas-cu12       12.3.4.1
nvidia-cuda-nvrtc-cu12   12.3.107
nvidia-cuda-runtime-cu12 12.3.101
nvidia-cudnn-cu12        8.9.7.29
packaging                23.2
parameterized            0.9.0
pathspec                 0.12.1
Pillow                   9.3.0
pip                      23.3.2
platformdirs             4.1.0
pluggy                   1.3.0
pre-commit               3.6.0
pytest                   7.4.4
PyYAML                   6.0.1
regex                    2023.12.25
requests                 2.28.1
ruff                     0.1.14
safetensors              0.4.1
setuptools               65.5.0
sympy                    1.11.1
tensorrt                 9.2.0.post12.dev5
tensorrt-bindings        9.2.0.post12.dev5
tensorrt-libs            9.2.0.post12.dev5
timm                     0.9.12
tokenizers               0.15.0
tomli                    2.0.1
torch                    2.3.0.dev20240120+cu121
torch-tensorrt           2.3.0.dev0+e7e291144
torchvision              0.18.0.dev20240120+cu121
tqdm                     4.66.1
transformers             4.36.2
typing_extensions        4.8.0
urllib3                  1.26.13
virtualenv               20.25.0
wheel                    0.42.0

If I clone to and then run: python setup.py install via CMD, this is what I get:

Microsoft Windows [Version 10.0.19045.3930]
(c) Microsoft Corporation. Alle Rechte vorbehalten.

C:\Users\{UserName}>cd C:\Users\{UserName}\source\repos\TensorRT

C:\Users\{UserName}\source\repos\TensorRT>python setup.py install
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\config\pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
  warnings.warn(msg, _BetaConfiguration)
running install
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
creating version file
running build
running build_py
copying py\torch_tensorrt\_version.py -> build\lib\torch_tensorrt
running install_lib
copying build\lib\torch_tensorrt\_version.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\_version.py to _version.cpython-310.pyc
running install_egg_info
running egg_info
writing py\torch_tensorrt.egg-info\PKG-INFO
writing dependency_links to py\torch_tensorrt.egg-info\dependency_links.txt
writing requirements to py\torch_tensorrt.egg-info\requires.txt
writing top-level names to py\torch_tensorrt.egg-info\top_level.txt
reading manifest file 'py\torch_tensorrt.egg-info\SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'py\torch_tensorrt.egg-info\SOURCES.txt'
removing 'C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt-2.3.0.dev0+e7e291144-py3.10.egg-info' (and everything under it)
Copying py\torch_tensorrt.egg-info to C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt-2.3.0.dev0+e7e291144-py3.10.egg-info
running install_scripts

C:\Users\{UserName}\source\repos\TensorRT>

Also if delete the folder in the site-packages folder torch_tensorrt and torch_tensorrt-2.3.0.dev0+e7e92911-py3.10.egg-info and run the code again. This happens and the folder are created there again:

Microsoft Windows [Version 10.0.19045.3930]
(c) Microsoft Corporation. Alle Rechte vorbehalten.

C:\Users\{UserName}>cd C:\Users\{UserName}\source\repos\TensorRT

C:\Users\{UserName}\source\repos\TensorRT>python setup.py install
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\config\pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
  warnings.warn(msg, _BetaConfiguration)
running install
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
creating version file
running build
running build_py
copying py\torch_tensorrt\_version.py -> build\lib\torch_tensorrt
running install_lib
copying build\lib\torch_tensorrt\_version.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\_version.py to _version.cpython-310.pyc
running install_egg_info
running egg_info
writing py\torch_tensorrt.egg-info\PKG-INFO
writing dependency_links to py\torch_tensorrt.egg-info\dependency_links.txt
writing requirements to py\torch_tensorrt.egg-info\requires.txt
writing top-level names to py\torch_tensorrt.egg-info\top_level.txt
reading manifest file 'py\torch_tensorrt.egg-info\SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'py\torch_tensorrt.egg-info\SOURCES.txt'
removing 'C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt-2.3.0.dev0+e7e291144-py3.10.egg-info' (and everything under it)
Copying py\torch_tensorrt.egg-info to C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt-2.3.0.dev0+e7e291144-py3.10.egg-info
running install_scripts

C:\Users\{UserName}\source\repos\TensorRT>python setup.py install
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\config\pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
  warnings.warn(msg, _BetaConfiguration)
running install
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
creating version file
running build
running build_py
copying py\torch_tensorrt\_version.py -> build\lib\torch_tensorrt
running install_lib
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\backend
copying build\lib\torch_tensorrt\dynamo\backend\backends.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\backend
copying build\lib\torch_tensorrt\dynamo\backend\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\backend
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion
copying build\lib\torch_tensorrt\dynamo\conversion\aten_ops_converters.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion
copying build\lib\torch_tensorrt\dynamo\conversion\converter_utils.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\activation
copying build\lib\torch_tensorrt\dynamo\conversion\impl\activation\base.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\activation
copying build\lib\torch_tensorrt\dynamo\conversion\impl\activation\ops.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\activation
copying build\lib\torch_tensorrt\dynamo\conversion\impl\activation\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\activation
copying build\lib\torch_tensorrt\dynamo\conversion\impl\addmm.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\attention.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\cast.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\cat.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\condition
copying build\lib\torch_tensorrt\dynamo\conversion\impl\condition\ops.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\condition
copying build\lib\torch_tensorrt\dynamo\conversion\impl\condition\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\condition
copying build\lib\torch_tensorrt\dynamo\conversion\impl\conv.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\deconv.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\elementwise
copying build\lib\torch_tensorrt\dynamo\conversion\impl\elementwise\base.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\elementwise
copying build\lib\torch_tensorrt\dynamo\conversion\impl\elementwise\ops.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\elementwise
copying build\lib\torch_tensorrt\dynamo\conversion\impl\elementwise\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\elementwise
copying build\lib\torch_tensorrt\dynamo\conversion\impl\embedding.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\grid.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\linear.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\matmul.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\normalization
copying build\lib\torch_tensorrt\dynamo\conversion\impl\normalization\ops.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\normalization
copying build\lib\torch_tensorrt\dynamo\conversion\impl\normalization\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\normalization
copying build\lib\torch_tensorrt\dynamo\conversion\impl\pad.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\permutation.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\pool.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\reduce.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\select.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\shape.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\shuffle.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\slice
copying build\lib\torch_tensorrt\dynamo\conversion\impl\slice\base.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\slice
copying build\lib\torch_tensorrt\dynamo\conversion\impl\slice\ops.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\slice
copying build\lib\torch_tensorrt\dynamo\conversion\impl\slice\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\slice
copying build\lib\torch_tensorrt\dynamo\conversion\impl\split.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\squeeze.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\topk.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\unary
copying build\lib\torch_tensorrt\dynamo\conversion\impl\unary\base.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\unary
copying build\lib\torch_tensorrt\dynamo\conversion\impl\unary\ops.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\unary
copying build\lib\torch_tensorrt\dynamo\conversion\impl\unary\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\unary
copying build\lib\torch_tensorrt\dynamo\conversion\impl\unsqueeze.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\upsample.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\impl\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl
copying build\lib\torch_tensorrt\dynamo\conversion\ops_evaluators.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion
copying build\lib\torch_tensorrt\dynamo\conversion\prims_ops_converters.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion
copying build\lib\torch_tensorrt\dynamo\conversion\truncate_long_and_double.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion
copying build\lib\torch_tensorrt\dynamo\conversion\_conversion.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion
copying build\lib\torch_tensorrt\dynamo\conversion\_ConversionContext.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion
copying build\lib\torch_tensorrt\dynamo\conversion\_ConverterRegistry.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion
copying build\lib\torch_tensorrt\dynamo\conversion\_TRTInterpreter.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion
copying build\lib\torch_tensorrt\dynamo\conversion\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes
copying build\lib\torch_tensorrt\dynamo\lowering\passes\constant_folding.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes
copying build\lib\torch_tensorrt\dynamo\lowering\passes\fuse_prims_broadcast.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes
copying build\lib\torch_tensorrt\dynamo\lowering\passes\lower_linear.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes
copying build\lib\torch_tensorrt\dynamo\lowering\passes\lower_scaled_dot_product_attention.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes
copying build\lib\torch_tensorrt\dynamo\lowering\passes\pass_manager.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes
copying build\lib\torch_tensorrt\dynamo\lowering\passes\pass_utils.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes
copying build\lib\torch_tensorrt\dynamo\lowering\passes\remove_input_alias_fixing_clones.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes
copying build\lib\torch_tensorrt\dynamo\lowering\passes\repair_input_as_output.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes
copying build\lib\torch_tensorrt\dynamo\lowering\passes\replace_max_pool_with_indices.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes
copying build\lib\torch_tensorrt\dynamo\lowering\passes\view_to_reshape.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes
copying build\lib\torch_tensorrt\dynamo\lowering\passes\_aten_lowering_pass.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes
copying build\lib\torch_tensorrt\dynamo\lowering\passes\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes
copying build\lib\torch_tensorrt\dynamo\lowering\_decompositions.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering
copying build\lib\torch_tensorrt\dynamo\lowering\_decomposition_groups.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering
copying build\lib\torch_tensorrt\dynamo\lowering\_fusers.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering
copying build\lib\torch_tensorrt\dynamo\lowering\_repair_input_aliasing.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering
copying build\lib\torch_tensorrt\dynamo\lowering\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\partitioning
copying build\lib\torch_tensorrt\dynamo\partitioning\common.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\partitioning
copying build\lib\torch_tensorrt\dynamo\partitioning\_adjacency_partitioner.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\partitioning
copying build\lib\torch_tensorrt\dynamo\partitioning\_global_partitioner.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\partitioning
copying build\lib\torch_tensorrt\dynamo\partitioning\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\partitioning
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\runtime
copying build\lib\torch_tensorrt\dynamo\runtime\tools.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\runtime
copying build\lib\torch_tensorrt\dynamo\runtime\_PythonTorchTensorRTModule.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\runtime
copying build\lib\torch_tensorrt\dynamo\runtime\_TorchTensorRTModule.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\runtime
copying build\lib\torch_tensorrt\dynamo\runtime\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\runtime
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\tools
copying build\lib\torch_tensorrt\dynamo\tools\opset_coverage.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\tools
copying build\lib\torch_tensorrt\dynamo\tools\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\tools
copying build\lib\torch_tensorrt\dynamo\utils.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo
copying build\lib\torch_tensorrt\dynamo\_compiler.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo
copying build\lib\torch_tensorrt\dynamo\_defaults.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo
copying build\lib\torch_tensorrt\dynamo\_DryRunTracker.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo
copying build\lib\torch_tensorrt\dynamo\_exporter.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo
copying build\lib\torch_tensorrt\dynamo\_settings.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo
copying build\lib\torch_tensorrt\dynamo\_SourceIR.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo
copying build\lib\torch_tensorrt\dynamo\_tracer.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo
copying build\lib\torch_tensorrt\dynamo\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
copying build\lib\torch_tensorrt\fx\converters\acc_ops_converters.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
copying build\lib\torch_tensorrt\fx\converters\adaptive_avgpool.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
copying build\lib\torch_tensorrt\fx\converters\add.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
copying build\lib\torch_tensorrt\fx\converters\aten_ops_converters.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
copying build\lib\torch_tensorrt\fx\converters\batchnorm.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
copying build\lib\torch_tensorrt\fx\converters\converter_utils.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\impl
copying build\lib\torch_tensorrt\fx\converters\impl\activation.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\impl
copying build\lib\torch_tensorrt\fx\converters\impl\convolution.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\impl
copying build\lib\torch_tensorrt\fx\converters\impl\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\impl
copying build\lib\torch_tensorrt\fx\converters\linear.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
copying build\lib\torch_tensorrt\fx\converters\maxpool.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
copying build\lib\torch_tensorrt\fx\converters\mul.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
copying build\lib\torch_tensorrt\fx\converters\nn_ops_converters.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
copying build\lib\torch_tensorrt\fx\converters\quantization.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
copying build\lib\torch_tensorrt\fx\converters\transformation.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
copying build\lib\torch_tensorrt\fx\converters\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters
copying build\lib\torch_tensorrt\fx\converter_registry.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx
copying build\lib\torch_tensorrt\fx\diagnostics.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx
copying build\lib\torch_tensorrt\fx\fx2trt.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx
copying build\lib\torch_tensorrt\fx\input_tensor_spec.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx
copying build\lib\torch_tensorrt\fx\lower.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx
copying build\lib\torch_tensorrt\fx\lower_setting.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx
copying build\lib\torch_tensorrt\fx\observer.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes
copying build\lib\torch_tensorrt\fx\passes\graph_opts.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes
copying build\lib\torch_tensorrt\fx\passes\lower_basic_pass.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes
copying build\lib\torch_tensorrt\fx\passes\lower_basic_pass_aten.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes
copying build\lib\torch_tensorrt\fx\passes\lower_pass_manager_builder.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes
copying build\lib\torch_tensorrt\fx\passes\pass_utils.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes
copying build\lib\torch_tensorrt\fx\passes\remove_duplicate_output_args.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes
copying build\lib\torch_tensorrt\fx\passes\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools
copying build\lib\torch_tensorrt\fx\tools\common_fx2trt.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools
copying build\lib\torch_tensorrt\fx\tools\engine_layer_visualize.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools
copying build\lib\torch_tensorrt\fx\tools\graph_util.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools
copying build\lib\torch_tensorrt\fx\tools\model_packager.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools
copying build\lib\torch_tensorrt\fx\tools\node_profiler.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools
copying build\lib\torch_tensorrt\fx\tools\tensor_prop.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools
copying build\lib\torch_tensorrt\fx\tools\timing_cache_utils.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools
copying build\lib\torch_tensorrt\fx\tools\trt_minimizer.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools
copying build\lib\torch_tensorrt\fx\tools\trt_profiler_sorted.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools
copying build\lib\torch_tensorrt\fx\tools\trt_splitter.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools
copying build\lib\torch_tensorrt\fx\tools\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer
copying build\lib\torch_tensorrt\fx\tracer\acc_tracer\acc_normalizer.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer
copying build\lib\torch_tensorrt\fx\tracer\acc_tracer\acc_ops.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer
copying build\lib\torch_tensorrt\fx\tracer\acc_tracer\acc_op_properties.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer
copying build\lib\torch_tensorrt\fx\tracer\acc_tracer\acc_shape_prop.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer
copying build\lib\torch_tensorrt\fx\tracer\acc_tracer\acc_tracer.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer
copying build\lib\torch_tensorrt\fx\tracer\acc_tracer\acc_utils.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer
copying build\lib\torch_tensorrt\fx\tracer\acc_tracer\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\dispatch_tracer
copying build\lib\torch_tensorrt\fx\tracer\dispatch_tracer\aten_tracer.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\dispatch_tracer
copying build\lib\torch_tensorrt\fx\tracer\dispatch_tracer\tracer.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\dispatch_tracer
copying build\lib\torch_tensorrt\fx\tracer\dispatch_tracer\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\dispatch_tracer
copying build\lib\torch_tensorrt\fx\tracer\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer
copying build\lib\torch_tensorrt\fx\trt_module.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx
copying build\lib\torch_tensorrt\fx\types.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx
copying build\lib\torch_tensorrt\fx\utils.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx
copying build\lib\torch_tensorrt\fx\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx
creating C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\runtime
copying build\lib\torch_tensorrt\runtime\multi_device_safe_mode.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\runtime
copying build\lib\torch_tensorrt\runtime\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\runtime
copying build\lib\torch_tensorrt\_compile.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt
copying build\lib\torch_tensorrt\_Device.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt
copying build\lib\torch_tensorrt\_enums.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt
copying build\lib\torch_tensorrt\_Input.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt
copying build\lib\torch_tensorrt\_utils.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt
copying build\lib\torch_tensorrt\_version.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt
copying build\lib\torch_tensorrt\__init__.py -> C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\backend\backends.py to backends.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\backend\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\aten_ops_converters.py to aten_ops_converters.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\converter_utils.py to converter_utils.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\activation\base.py to base.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\activation\ops.py to ops.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\activation\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\addmm.py to addmm.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\attention.py to attention.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\cast.py to cast.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\cat.py to cat.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\condition\ops.py to ops.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\condition\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\conv.py to conv.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\deconv.py to deconv.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\elementwise\base.py to base.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\elementwise\ops.py to ops.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\elementwise\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\embedding.py to embedding.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\grid.py to grid.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\linear.py to linear.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\matmul.py to matmul.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\normalization\ops.py to ops.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\normalization\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\pad.py to pad.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\permutation.py to permutation.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\pool.py to pool.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\reduce.py to reduce.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\select.py to select.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\shape.py to shape.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\shuffle.py to shuffle.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\slice\base.py to base.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\slice\ops.py to ops.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\slice\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\split.py to split.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\squeeze.py to squeeze.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\topk.py to topk.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\unary\base.py to base.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\unary\ops.py to ops.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\unary\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\unsqueeze.py to unsqueeze.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\upsample.py to upsample.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\impl\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\ops_evaluators.py to ops_evaluators.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\prims_ops_converters.py to prims_ops_converters.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\truncate_long_and_double.py to truncate_long_and_double.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\_conversion.py to _conversion.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\_ConversionContext.py to _ConversionContext.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\_ConverterRegistry.py to _ConverterRegistry.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\_TRTInterpreter.py to _TRTInterpreter.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\conversion\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes\constant_folding.py to constant_folding.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes\fuse_prims_broadcast.py to fuse_prims_broadcast.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes\lower_linear.py to lower_linear.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes\lower_scaled_dot_product_attention.py to lower_scaled_dot_product_attention.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes\pass_manager.py to pass_manager.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes\pass_utils.py to pass_utils.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes\remove_input_alias_fixing_clones.py to remove_input_alias_fixing_clones.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes\repair_input_as_output.py to repair_input_as_output.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes\replace_max_pool_with_indices.py to replace_max_pool_with_indices.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes\view_to_reshape.py to view_to_reshape.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes\_aten_lowering_pass.py to _aten_lowering_pass.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\passes\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\_decompositions.py to _decompositions.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\_decomposition_groups.py to _decomposition_groups.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\_fusers.py to _fusers.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\_repair_input_aliasing.py to _repair_input_aliasing.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\lowering\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\partitioning\common.py to common.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\partitioning\_adjacency_partitioner.py to _adjacency_partitioner.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\partitioning\_global_partitioner.py to _global_partitioner.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\partitioning\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\runtime\tools.py to tools.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\runtime\_PythonTorchTensorRTModule.py to _PythonTorchTensorRTModule.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\runtime\_TorchTensorRTModule.py to _TorchTensorRTModule.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\runtime\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\tools\opset_coverage.py to opset_coverage.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\tools\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\utils.py to utils.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\_compiler.py to _compiler.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\_defaults.py to _defaults.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\_DryRunTracker.py to _DryRunTracker.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\_exporter.py to _exporter.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\_settings.py to _settings.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\_SourceIR.py to _SourceIR.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\_tracer.py to _tracer.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\dynamo\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\acc_ops_converters.py to acc_ops_converters.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\adaptive_avgpool.py to adaptive_avgpool.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\add.py to add.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\aten_ops_converters.py to aten_ops_converters.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\batchnorm.py to batchnorm.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\converter_utils.py to converter_utils.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\impl\activation.py to activation.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\impl\convolution.py to convolution.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\impl\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\linear.py to linear.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\maxpool.py to maxpool.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\mul.py to mul.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\nn_ops_converters.py to nn_ops_converters.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\quantization.py to quantization.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\transformation.py to transformation.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converters\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\converter_registry.py to converter_registry.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\diagnostics.py to diagnostics.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\fx2trt.py to fx2trt.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\input_tensor_spec.py to input_tensor_spec.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\lower.py to lower.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\lower_setting.py to lower_setting.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\observer.py to observer.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes\graph_opts.py to graph_opts.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes\lower_basic_pass.py to lower_basic_pass.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes\lower_basic_pass_aten.py to lower_basic_pass_aten.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes\lower_pass_manager_builder.py to lower_pass_manager_builder.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes\pass_utils.py to pass_utils.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes\remove_duplicate_output_args.py to remove_duplicate_output_args.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\passes\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools\common_fx2trt.py to common_fx2trt.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools\engine_layer_visualize.py to engine_layer_visualize.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools\graph_util.py to graph_util.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools\model_packager.py to model_packager.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools\node_profiler.py to node_profiler.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools\tensor_prop.py to tensor_prop.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools\timing_cache_utils.py to timing_cache_utils.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools\trt_minimizer.py to trt_minimizer.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools\trt_profiler_sorted.py to trt_profiler_sorted.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools\trt_splitter.py to trt_splitter.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tools\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer\acc_normalizer.py to acc_normalizer.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer\acc_ops.py to acc_ops.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer\acc_op_properties.py to acc_op_properties.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer\acc_shape_prop.py to acc_shape_prop.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer\acc_tracer.py to acc_tracer.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer\acc_utils.py to acc_utils.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\acc_tracer\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\dispatch_tracer\aten_tracer.py to aten_tracer.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\dispatch_tracer\tracer.py to tracer.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\dispatch_tracer\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\tracer\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\trt_module.py to trt_module.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\types.py to types.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\utils.py to utils.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\fx\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\runtime\multi_device_safe_mode.py to multi_device_safe_mode.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\runtime\__init__.py to __init__.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\_compile.py to _compile.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\_Device.py to _Device.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\_enums.py to _enums.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\_Input.py to _Input.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\_utils.py to _utils.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\_version.py to _version.cpython-310.pyc
byte-compiling C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt\__init__.py to __init__.cpython-310.pyc
running install_egg_info
running egg_info
writing py\torch_tensorrt.egg-info\PKG-INFO
writing dependency_links to py\torch_tensorrt.egg-info\dependency_links.txt
writing requirements to py\torch_tensorrt.egg-info\requires.txt
writing top-level names to py\torch_tensorrt.egg-info\top_level.txt
reading manifest file 'py\torch_tensorrt.egg-info\SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'py\torch_tensorrt.egg-info\SOURCES.txt'
Copying py\torch_tensorrt.egg-info to C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\torch_tensorrt-2.3.0.dev0+e7e291144-py3.10.egg-info
running install_scripts

C:\Users\{UserName}\source\repos\TensorRT>

So what do we now?

Jan 24 '24 15:01 jensdraht1999

This will happen in Powershell if import is done:

PS C:\Users{UserName}> python Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information.

import torch; import torch_tensorrt C:\Users{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\utils\generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. _torch_pytree._register_pytree_node( C:\Users{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torch_tensorrt_Device.py:19: UserWarning: Unable to import torchscript frontend core and torch-tensorrt runtime. Some dependent features may be unavailable. warnings.warn(

Jan 24 '24 15:01 jensdraht1999

WAIT A MINUTE:

Could it be, that regular GPUs like RTX 3060 LAPTOP are not supported by Tensorrt at all?

Jan 24 '24 15:01 jensdraht1999

I thought gs-olive had a typo in https://github.com/pytorch/TensorRT/issues/2577#issuecomment-1897730589. It should be optimized_model = torch.compile(model, backend="eager"), not optimized_model = torch_tensorrt.compile(model, backend="eager").

import torch
import torchvision.models as models

model = models.resnet18(pretrained=True).half().eval().to("cuda")
inputs = [torch.randn((1, 3, 224, 224)).to("cuda").half()]
optimized_model = torch.compile(model, backend="eager")
print(optimized_model(*inputs))

Jan 24 '24 16:01 HolyWu

It has worked with the code you have provided. Thank you. @HolyWu

Here is the what is the output:

Microsoft Windows [Version 10.0.19045.3930]
(c) Microsoft Corporation. Alle Rechte vorbehalten.

C:\Users\{UserName}>C:\Users\{UserName}\Desktop\whl\2.py
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\utils\generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  _torch_pytree._register_pytree_node(
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
C:\Users\{UserName}\AppData\Local\Programs\Python\Python310\lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
tensor([[ 1.0977e+00,  2.5957e+00,  2.9277e+00,  2.9453e+00,  4.3242e+00,
          3.6836e+00,  3.9395e+00,  3.5767e-01, -8.9941e-01, -6.6797e-01,
         -7.0508e-01,  1.2744e+00,  1.7505e-01,  1.2754e+00,  1.5352e+00,
          7.6611e-01, -7.4854e-01, -2.7759e-01,  1.2705e+00,  3.0103e-01,
         -5.8350e-02, -2.9272e-01,  1.4014e+00,  4.0552e-01, -1.9031e-01,
          2.1399e-01,  8.3545e-01,  6.3867e-01,  3.1055e-01,  7.8662e-01,
          1.3662e+00,  3.8574e-01, -6.9531e-01,  2.6582e+00,  3.2012e+00,
          6.6846e-01,  4.0527e-01,  4.0747e-01,  3.3057e-01,  1.9170e+00,
          1.6221e+00,  6.8115e-01,  9.9951e-01,  4.8120e-01,  1.3926e+00,
          2.6367e-01,  1.7412e+00, -1.1719e+00,  1.6943e+00,  6.7334e-01,
          1.8330e+00, -1.7939e+00,  6.2500e-01,  5.9521e-01, -1.8408e-01,
          1.3535e+00, -8.2666e-01,  3.0469e-01,  1.9014e+00,  2.4866e-01,
          1.4521e+00, -1.1499e-01,  2.1561e-02, -1.1504e+00,  3.0518e-01,
          4.3438e+00,  5.5725e-02, -2.7368e-01,  3.2568e-01,  1.9385e+00,
          1.6504e+00,  2.1035e+00,  5.1904e-01,  8.2959e-01,  6.6504e-01,
          1.9297e+00,  8.6328e-01,  1.4189e+00,  1.3486e+00,  2.4277e+00,
         -1.5957e+00,  1.1562e+00, -2.5269e-01, -2.8730e+00,  5.9424e-01,
          7.2021e-01, -5.2148e-01, -4.9268e-01, -2.8589e-01,  1.5371e+00,
         -1.2080e+00, -1.1064e+00,  5.2246e-01, -4.7583e-01,  3.2788e-01,
         -2.4141e+00,  1.1993e-01, -8.3740e-01, -1.5615e+00,  1.2524e-01,
         -1.2732e-01, -1.9434e+00, -1.5410e+00,  2.0984e-01,  4.2871e-01,
         -2.2095e-01, -1.1926e-01,  6.4531e+00,  2.2832e+00,  2.4609e+00,
          1.8496e+00,  4.0039e+00,  2.6035e+00,  1.8965e+00,  1.9883e+00,
          1.4610e-02,  9.0234e-01,  1.3057e+00,  4.6729e-01,  3.9331e-01,
          5.5713e-01,  1.2622e-01,  6.1035e-01,  1.8857e+00,  2.4961e+00,
          1.8994e+00,  2.5059e+00, -6.2195e-02,  1.3013e-01,  9.8291e-01,
         -2.0239e-01,  2.4390e-01,  1.7998e+00,  3.2251e-01,  5.9473e-01,
         -7.1143e-01, -1.3359e+00,  2.2888e-01,  5.2246e-01, -3.9453e-01,
         -2.1021e-01,  1.4319e-01, -3.4180e-01,  5.2148e-01,  2.3230e-01,
          5.0293e-01, -5.8350e-01,  3.1201e-01,  7.1631e-01,  2.1172e+00,
          1.2432e+00, -2.9224e-01, -2.5527e+00,  4.9756e-01, -7.1582e-01,
         -5.0690e-02, -1.3408e+00, -8.1152e-01, -7.7637e-01, -8.9795e-01,
         -2.5566e+00, -7.3059e-02,  6.5430e-02, -7.1680e-01, -1.0459e+00,
         -1.4453e+00, -1.0342e+00, -2.2910e+00, -1.0391e+00, -1.1230e+00,
         -1.8203e+00, -1.0797e-01, -1.5186e+00, -1.9409e-01, -1.1855e+00,
         -2.6348e+00, -1.3877e+00, -2.3027e+00,  8.3740e-01, -1.0830e+00,
         -5.2783e-01, -8.6816e-01, -9.4727e-01, -1.1318e+00, -1.3525e+00,
         -9.4482e-01, -1.7070e+00, -4.6753e-01, -2.5898e+00, -7.5293e-01,
         -1.2969e+00, -1.4766e+00, -9.5264e-01, -1.8721e+00, -2.3066e+00,
          1.3086e-01, -1.0898e+00, -2.6055e+00, -2.2012e+00, -9.4434e-01,
         -2.7383e+00,  9.8999e-02, -1.0615e+00,  1.2744e+00,  9.3555e-01,
         -1.7930e+00, -1.8076e+00, -2.5928e-01, -1.1924e+00,  1.0046e-01,
         -1.9707e+00, -1.3594e+00, -1.1045e+00, -5.8252e-01, -2.1367e+00,
         -1.1055e+00, -1.1748e+00, -3.3618e-01, -7.0215e-01, -1.3945e+00,
         -2.8984e+00, -1.2021e+00, -1.8428e+00, -2.6929e-01, -5.8594e-01,
         -1.4053e+00, -1.8711e+00, -1.9717e+00, -2.7637e+00, -9.6436e-01,
         -3.6011e-01, -1.1240e+00, -2.0959e-01, -3.0273e+00, -8.7695e-01,
         -1.4375e+00, -8.9404e-01, -1.0234e+00, -8.5693e-01, -1.4072e+00,
         -1.1689e+00, -1.4766e+00, -4.6631e-01, -1.2041e+00, -4.1797e+00,
         -3.8965e-01, -1.8516e+00, -1.2158e+00, -3.7720e-01, -1.5732e+00,
         -6.2354e-01, -5.2490e-01, -3.0254e+00, -5.4688e-01, -5.6982e-01,
         -2.3516e+00, -2.4004e+00, -1.0293e+00, -6.5625e-01,  3.2684e-02,
         -1.9229e+00, -1.6113e+00, -3.4121e+00, -3.7842e-01, -1.2832e+00,
         -5.8887e-01, -1.0410e+00, -2.1445e+00, -2.3770e+00, -1.2520e+00,
         -5.8398e-01, -1.7959e+00,  8.3801e-02, -1.8730e+00, -1.4395e+00,
         -2.3340e+00, -2.3828e+00, -8.3643e-01, -1.4102e+00,  6.6064e-01,
          3.7158e-01,  4.6631e-01,  7.3242e-01, -1.0278e-01,  5.5908e-01,
         -2.0190e-01, -1.1641e+00, -9.8877e-01, -6.8408e-01, -1.3857e+00,
         -2.6191e+00, -1.7529e+00, -8.7305e-01, -1.8965e+00, -1.3340e+00,
         -2.0020e-01,  4.5825e-01, -2.9941e+00,  6.4062e-01, -7.8809e-01,
          1.1432e-01,  1.3848e+00,  6.0107e-01,  1.5820e+00, -6.5247e-02,
          3.6255e-02,  7.3877e-01,  9.7266e-01,  6.1084e-01,  2.1167e-01,
          3.0000e+00,  1.3105e+00,  1.8643e+00,  1.9268e+00,  2.9258e+00,
          2.5234e+00,  1.6104e+00,  2.6523e+00,  3.3184e+00,  1.1943e+00,
          4.8755e-01, -3.2520e-01, -1.2656e+00, -3.0225e-01,  1.6631e+00,
          6.2637e-03,  4.6851e-01,  3.7051e+00,  1.3770e+00,  3.0938e+00,
          1.6760e-01,  3.3667e-01, -2.6538e-01,  7.9346e-02,  3.2275e-01,
          4.3286e-01, -2.3804e-01, -4.3774e-01, -4.1351e-02, -1.9609e+00,
         -1.7285e+00, -1.8770e+00, -1.3252e+00, -7.5000e-01, -1.1774e-01,
         -1.4834e+00, -2.2363e+00, -1.3203e+00, -1.5596e+00, -7.0020e-01,
         -2.2864e-01, -1.7383e+00, -1.1133e+00, -6.5820e-01, -1.6035e+00,
         -2.3613e+00,  3.1494e-01, -3.6426e-01,  2.1240e-01,  5.1575e-02,
         -4.9243e-01,  1.9980e+00,  1.9653e-01,  1.2529e+00, -1.2793e+00,
         -7.5537e-01, -2.3242e+00, -1.5264e+00, -1.3252e+00, -1.1777e+00,
          6.4209e-02, -2.5801e+00, -1.8984e+00, -3.5767e-01,  7.4219e-01,
         -8.0225e-01, -5.4736e-01, -1.6187e-01, -1.2881e+00, -1.1865e+00,
         -8.4473e-01, -7.0850e-01, -4.7021e-01,  2.4707e-01, -1.1416e+00,
         -1.6260e+00, -1.3623e+00, -1.4531e+00, -1.8467e+00,  6.8799e-01,
          2.9473e+00,  2.4746e+00,  1.8848e+00,  1.7402e+00,  1.7656e+00,
          1.9551e+00,  2.7012e+00,  2.4707e+00, -1.0919e-01, -4.3481e-01,
         -1.6240e+00, -1.0742e-01,  1.2666e+00, -4.3408e-01, -6.9092e-01,
          1.1934e+00, -4.7827e-01, -2.2578e+00, -2.1660e+00,  1.6143e+00,
          1.3174e+00, -2.4011e-01,  1.2139e+00,  1.8030e-01, -7.4890e-02,
         -1.9629e+00, -4.4238e-01,  1.0049e+00,  1.2900e+00,  1.9473e+00,
         -1.9470e-02,  1.2637e+00,  7.5928e-01, -2.2715e+00, -1.7344e+00,
          1.0488e+00,  6.3599e-02,  7.5732e-01,  1.4746e-01,  2.2305e+00,
         -4.4849e-01, -2.0020e+00, -2.4768e-01,  9.0088e-01, -6.3037e-01,
         -1.0586e+00, -1.0068e+00,  9.1113e-01,  5.8252e-01, -2.2207e+00,
          2.4727e+00,  2.0898e+00, -6.2256e-01, -6.6406e-02, -8.1885e-01,
         -3.4229e-01,  1.7549e+00,  4.2041e-01,  7.0117e-01, -6.5735e-02,
         -2.4238e+00,  7.2363e-01, -7.5049e-01, -8.7646e-01, -1.2002e+00,
          2.9746e+00,  3.6328e-01, -5.8472e-02,  1.8691e+00,  7.0020e-01,
          1.3408e+00,  1.7249e-01,  8.6182e-01,  2.2246e+00,  2.8735e-01,
         -1.5615e+00, -1.9434e+00, -2.3027e+00,  1.5146e+00, -2.1167e-01,
         -4.1968e-01, -6.9275e-02, -3.0078e-01, -4.2310e-01,  1.7949e+00,
         -6.8604e-01, -3.7285e+00, -1.0791e+00,  5.2881e-01,  6.8652e-01,
          8.1982e-01,  2.4707e+00, -2.0874e-01,  2.0176e+00,  2.8137e-02,
          6.5002e-02, -2.2766e-01, -5.5420e-01,  3.4238e+00,  3.5098e+00,
          1.5889e+00,  1.7273e-01,  5.8301e-01, -2.1074e+00,  8.7207e-01,
         -1.6396e+00, -3.8062e-01,  1.6201e+00,  5.7031e-01, -2.0703e-01,
         -2.6306e-02, -4.5728e-01,  1.1768e+00, -5.1172e-01, -3.3691e-01,
         -1.2317e-01,  2.0156e+00,  1.8145e+00,  1.2148e+00, -1.4834e+00,
         -1.7246e+00, -1.8730e+00,  1.0488e+00, -2.5293e-01,  1.0834e-01,
          3.7628e-02, -1.7812e+00,  4.9316e-01, -4.2017e-01, -4.6411e-01,
         -2.2734e+00, -1.2510e+00,  2.6123e-01,  1.5361e+00, -9.5117e-01,
         -7.5867e-02, -1.6426e+00, -1.2422e+00,  1.0358e-01, -2.5586e-01,
          1.2188e+00, -3.6768e-01, -1.9277e+00,  7.7197e-01, -1.7256e+00,
         -3.4058e-01, -5.8545e-01, -4.1870e-01,  2.1641e+00,  3.0312e+00,
         -7.4902e-01, -1.4038e-01,  1.0039e+00,  5.9766e-01,  7.2998e-01,
          1.0488e+00,  2.4353e-01, -2.4023e+00, -1.8184e+00,  1.8008e+00,
         -3.1973e+00, -1.1387e+00,  6.2158e-01, -4.8535e-01, -1.1104e+00,
         -2.0664e+00, -5.8740e-01,  1.0508e+00,  1.0498e-01,  1.6426e+00,
          1.2988e+00, -1.1484e+00,  3.3594e+00,  1.5801e+00, -1.4863e+00,
         -1.0674e+00,  1.9177e-01,  8.1689e-01, -1.9287e+00, -9.9170e-01,
          6.2305e-01, -9.0967e-01,  3.4155e-01, -9.9121e-01,  2.7715e+00,
         -1.8203e+00, -1.0645e+00,  3.5913e-01, -4.6265e-01, -3.3447e-01,
         -4.3774e-01,  8.0713e-01, -2.2617e+00, -1.0996e+00, -5.7031e-01,
         -1.0752e+00, -1.9199e+00,  1.6055e+00, -7.1045e-01, -7.8125e-02,
         -1.2812e+00, -1.3733e-01,  5.6671e-02,  6.5479e-01,  1.0809e-01,
         -8.9355e-01,  5.8350e-01,  1.3906e+00,  3.3398e-01,  1.4561e+00,
          2.2441e+00, -7.5732e-01, -2.6147e-01, -2.7852e+00,  1.5566e+00,
          1.1104e+00,  2.2285e+00, -1.0664e+00,  2.2969e+00, -1.7822e+00,
          2.1802e-01,  5.1094e+00, -2.1543e+00,  1.5117e+00, -1.4912e+00,
          1.1982e+00,  2.8457e+00, -1.7617e+00,  3.5229e-01,  7.3682e-01,
          8.9160e-01, -2.6196e-01,  2.8223e+00,  8.0859e-01, -9.0430e-01,
         -1.4727e+00,  3.3350e-01, -1.2910e+00, -6.5063e-02, -5.0391e-01,
          1.0199e-01, -2.8540e-01,  1.1738e+00,  1.6318e+00, -2.9199e+00,
          1.4209e-01, -5.0439e-01,  1.3604e+00,  4.0894e-01,  3.4741e-01,
          3.4219e+00, -8.7939e-01, -1.7646e+00,  6.1670e-01,  3.7578e+00,
         -9.2236e-01,  2.0801e+00,  3.2251e-01, -1.3809e+00,  9.1504e-01,
          1.2588e+00, -1.2891e+00, -1.5107e+00,  6.7480e-01, -1.5264e+00,
          7.2949e-01, -1.3252e+00,  1.1143e+00,  9.1357e-01,  2.4487e-01,
         -8.6865e-01, -1.5322e+00,  1.1064e+00,  5.3760e-01,  1.3438e+00,
         -1.1377e-01,  1.1318e+00,  8.1592e-01,  1.0322e+00,  4.2505e-01,
         -1.8721e+00, -2.8732e-02,  6.5723e-01,  1.6133e+00,  2.0840e+00,
         -2.0371e+00, -1.9785e+00,  2.8926e+00, -1.2715e+00,  1.7461e+00,
         -2.3047e-01,  9.5508e-01,  1.7188e+00,  3.3569e-01,  4.0674e-01,
          3.8794e-01, -4.6045e-01, -1.9961e+00,  1.0918e+00, -1.6562e+00,
         -3.2480e+00,  3.1714e-01,  8.8623e-01,  2.5757e-01, -4.5020e-01,
          3.1934e+00,  1.4600e+00, -1.3115e+00,  6.3916e-01, -9.0869e-01,
          7.8955e-01,  1.4453e+00, -5.3809e-01,  9.2529e-01,  1.9531e+00,
         -2.0195e+00,  3.8794e-01,  5.1904e-01, -5.9229e-01,  1.5308e-01,
          9.6436e-01,  1.0273e+00,  2.5293e+00, -2.6855e+00,  2.0391e+00,
         -1.3506e+00,  1.2373e+00, -2.3008e+00,  1.3457e+00,  4.5459e-01,
          2.3145e+00,  3.5400e-01,  3.7207e+00,  8.8672e-01, -4.6289e-01,
          1.5686e-01, -1.3965e+00,  1.8867e+00,  1.9238e+00, -2.2500e+00,
         -1.2291e-02,  4.0479e-01, -1.1680e+00,  3.0547e+00, -2.6270e+00,
          1.5107e+00,  1.8838e+00,  1.7412e+00, -8.7500e-01, -2.4727e+00,
          6.3379e-01,  2.7363e+00, -1.3115e+00,  7.8857e-01,  5.3320e-01,
         -7.6416e-01,  7.0508e-01,  1.9189e-01, -1.9373e-01,  1.6328e+00,
          5.5615e-01, -1.4121e+00,  2.2363e+00,  1.3623e+00, -5.9662e-02,
          1.0034e-01,  2.6440e-01, -1.8740e+00,  9.5215e-01,  3.2910e-01,
         -8.8037e-01,  1.0020e+00, -1.8604e+00,  1.8252e+00,  1.2817e-02,
         -3.9429e-01, -2.5898e+00,  2.0703e+00,  1.3342e-01,  2.0020e+00,
          1.5859e+00,  1.2520e+00,  1.0986e+00,  1.5996e+00,  7.7783e-01,
          9.6240e-01,  7.9883e-01,  1.3242e+00,  1.8835e-01,  2.6123e-01,
         -7.9443e-01, -4.3164e-01,  1.6113e+00,  3.1426e+00,  1.9736e+00,
         -2.1367e+00, -2.4011e-01, -2.8052e-01, -1.4727e+00, -1.7793e+00,
         -1.2500e+00,  1.5459e+00,  1.0215e+00, -5.3955e-01,  1.0576e+00,
          6.2451e-01, -3.3508e-02, -8.6572e-01, -7.8906e-01, -9.0088e-01,
         -1.9346e+00,  1.2285e+00, -9.6631e-01, -6.5674e-01,  9.4385e-01,
          1.4619e+00,  1.4697e+00, -1.5010e+00, -3.6426e-01,  1.1682e-01,
         -2.5410e+00,  1.1787e+00,  2.1460e-01, -3.0469e-01,  5.2490e-01,
          2.2949e+00, -8.1543e-02, -2.5195e+00,  2.7949e+00,  3.9624e-01,
         -1.8584e+00,  3.1372e-01, -1.3311e+00, -1.4209e-01,  5.6445e-01,
         -4.6997e-02,  7.8467e-01, -1.0947e+00,  2.9121e+00, -1.5557e+00,
         -1.4795e+00, -1.6660e+00,  9.5581e-02, -2.1472e-01, -2.4768e-01,
          1.7441e+00,  5.1318e-01,  7.5732e-01, -7.2510e-01,  6.2842e-01,
          3.3911e-01,  2.5952e-01,  6.9580e-01,  9.7314e-01,  3.4688e+00,
          4.6602e+00, -7.4829e-02, -2.5020e+00,  2.5342e-01, -9.0149e-02,
         -1.6870e-01,  1.4180e+00,  4.0039e+00,  5.8398e-01, -3.2013e-02,
          1.4434e+00, -3.4648e+00, -7.9736e-01,  1.2529e+00, -1.6373e-02,
         -2.3477e+00, -3.6816e-01, -4.6411e-01, -2.0007e-01, -1.2168e+00,
         -2.9062e+00, -9.2773e-01, -6.3330e-01,  5.8105e-01, -5.2686e-01,
          3.3789e-01, -6.3184e-01,  1.7383e+00,  1.0391e+00, -2.8945e+00,
         -3.4521e-01, -3.6353e-01, -8.4326e-01, -1.8838e+00,  3.0176e+00,
          2.3840e-01, -4.6295e-02,  7.5391e-01, -3.9429e-01,  1.1113e+00,
          1.8330e+00, -5.6787e-01, -1.2842e+00,  1.1572e+00, -5.6592e-01,
          4.4360e-01, -1.2695e+00,  2.5996e+00,  1.9355e+00, -5.8057e-01,
         -1.0654e+00, -4.7089e-02, -1.2627e+00,  2.2285e+00,  7.0333e-05,
          7.0654e-01, -1.5234e+00,  1.4580e+00, -3.2422e-01,  1.6494e+00,
         -5.3076e-01, -1.3037e+00,  1.0225e+00,  1.5107e+00, -2.7734e-01,
          3.2300e-01,  6.1328e-01, -3.1079e-01,  2.3711e+00, -2.0664e+00,
         -5.3662e-01, -7.9346e-01, -4.1064e-01, -5.9814e-01,  9.4727e-01,
          1.7461e+00,  5.3986e-02, -8.2275e-01, -1.8506e+00, -1.1699e+00,
         -1.2539e+00, -3.4180e+00, -2.3425e-01, -9.3506e-01,  1.1924e+00,
         -8.5547e-01, -5.6494e-01,  1.1560e-01, -1.5371e+00, -1.4717e+00,
         -8.4912e-01, -1.4395e+00,  1.3447e+00, -9.7217e-01, -1.3467e+00,
         -5.3320e-01, -2.7070e+00, -1.5215e+00, -1.3994e+00, -1.9795e+00,
         -5.9570e-01, -1.6279e+00, -4.8022e-01,  3.3667e-01, -3.1605e-03,
          1.0967e+00,  2.1641e+00,  8.6426e-01, -1.5605e+00, -1.6797e-01,
          1.2197e+00, -2.7090e+00, -1.3220e-01,  7.1875e-01, -1.1953e+00,
         -1.4980e+00,  2.7466e-01, -1.4648e+00,  4.5288e-02, -2.1797e+00,
         -3.4160e+00,  1.5430e-01,  1.0908e+00,  6.4209e-01, -3.5547e-01,
         -8.5547e-01,  4.0469e+00,  6.1462e-02,  3.1113e+00,  1.2451e+00,
          1.6895e+00,  1.1113e+00,  2.9668e+00,  2.0332e+00,  2.7246e-01,
          2.2109e+00,  7.5000e-01, -8.3923e-02,  3.5371e+00,  1.8193e+00,
          1.2119e+00,  3.3496e-01,  1.8877e+00,  1.4268e+00, -6.2744e-01,
          6.2061e-01, -5.1758e-01, -6.0645e-01, -3.8340e+00, -2.4160e+00,
         -1.7158e+00, -2.1582e+00,  3.5376e-01,  4.8682e-01,  1.0225e+00]],
       device='cuda:0', dtype=torch.float16, grad_fn=<AddmmBackward0>)

C:\Users\{UserName}>

Jan 24 '24 16:01 jensdraht1999

@HolyWu Just wanted to ask you, did it work for you, without deleting "C:\Users{UserName}\AppData\Local\Programs\Python\Python310\Lib\site-packages\nvidia\cudnn" folder?

Jan 24 '24 16:01 jensdraht1999

TensorRT TensorRT copied to clipboard

Can please somebody give a clear explanation of how to install torch-tensorrt on Windows?

❓ Question

Environment

Steps for Dynamo-Only, Python-Runtime-Only Windows build using windows_CI Branch

TensorRT
TensorRT copied to clipboard

Steps for Dynamo-Only, Python-Runtime-Only Windows build using `windows_CI` Branch