TensorRT
TensorRT copied to clipboard
pytorch
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
# Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change....
**Is your feature request related to a problem? Please describe.** Compare ONNX vs ATen subgraph perf **Describe the solution you'd like** **Describe alternatives you've considered** **Additional context**
## Bug Description This fixed temporary [file path](https://github.com/pytorch/TensorRT/blob/ca0765c444af2ddd6cc4c6d0361887f95f2208d9/tests/py/dynamo/models/test_export_serde.py#L20) used by all tests will cause racing condition if tests are running in parallel. ## Expected behavior Tests should be fine if...
**Is your feature request related to a problem? Please describe.** **Describe the solution you'd like** If I have multiple custom ops, I can create a plugin for each, but each...
…or folks locally # Description The lock file had a stale version of torch in it and some symbols got shifted around. Caused the C++ build to use latest but...
## Bug Description ## To Reproduce Steps to reproduce the behavior: 1. Install the torch_tensorrt wheels found at https://pypi.jetson-ai-lab.io/jp6/cu126 (2.8 for cu126) on a Jetson Orin Nano running Jetpack 6.2...
The PR addresses 1. Llama3 end to end example with complex graph lowering 2. Removal of hardcoded components of rotary embedding example
TRT-LLM installation tool for distributed 1. The download is to be done by only one GPU to avoid unnecessary downloads 2. Use of lock files in the tool for the...
**Is your feature request related to a problem? Please describe.** We support generating JIT QDP plugins automatically from PyTorch custom op registrations, but the performance makes them pretty unusable. Therefore...
## TL;DR ## Goal(s) ## Tasks ```[tasklist] ### Tasks ``` ## Additional context