TensorRT

TensorRT copied to clipboard

❓ [Question] How to solve this warning: Detected this engine is being instantitated in a multi-GPU system with multi-device safe mode disabled.

6

## ❓ Question I used Torch-TensorRT to compile the torchscript model in C++. When compiling or loading torchtrt model, it displays many warnings. ``` WARNING: [Torch-TensorRT] - Detected this engine...

demuxin

ci: Add release flag for nightly build tag

2

# Description - As is added in #2806 for C++ builds, enable `--release` flag for builds in `main` to enable correct nightly versioning on PyTorch index ## Type of change...

gs-olive

❓ [Question] Model inference is much slower after updating to TensorRT 9.3

1

## ❓ Question I have a VIT model for object detection. The model inference speed in the tensort 8.5 environment is 190ms per frame. However when I updated to TensorRT...

demuxin

🐛 [Bug] Nightly import fails

2

## Bug Description Installed nightly and got OSError: libnvinfer_plugin.so.10: cannot open shared object file: No such file or directory ## To Reproduce ``` pip install -U --pre torch_tensorrt --index-url https://download.pytorch.org/whl/nightly/cu121...

laikhtewari

**Is your feature request related to a problem? Please describe.** There are some cases where taking the extra step of wrapping a torch layer or pytorch custom op in a...

narendasan

Cherry-pick rand converter for release/2.3

apbose

# Description cherry pick of #2832 ## Type of change Please delete options that are not relevant and/or add your own. - Bug fix (non-breaking change which fixes an issue)...

zewenli98

🐛 [Bug] error: backend='torch_tensorrt' raised: TypeError: pybind11::init(): factory function returned nullptr

5

## Bug Description hi i see the following error - it looks like the torch.compile worked fine but when i invoke the prediction after that it errors out: ``` [INFO...

geraldstanje

# Description Cherry pick a previous commit. # Checklist: - [ ] My code follows the style guidelines of this project (You can use the linters) - [ ] I...

bowang007

# Description - Cherry-pick of #2821 ## Type of change - Cherry-pick # Checklist: - [ x ] My code follows the style guidelines of this project (You can use...

gs-olive