TensorRT
TensorRT copied to clipboard
pytorch
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
## Bug Description In real-world scenarios, user features are constantly changing, so I must use a list as the input for the forward function. but when I use list input,...
## Bug Description https://github.com/pytorch/TensorRT/blob/main/examples/dynamo/mutable_torchtrt_module_example.py I replaced hugging face whisper model instead of diffusion model ## To Reproduce import numpy as np import torch import torch_tensorrt as torch_trt from transformers import...
**Is your feature request related to a problem? Please describe.** Caching right now is weight agnostic, but at the cost of creating lower performance engines. **Describe the solution you'd like**...
## Bug Description Compile error when sd_unet model is running with dynamo backend. No compile error when torch_compile is used. This error happens after below commit. chore: Fixes required for...
# Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change....
## Bug Description FAILED runtime/test_002_lazy_engine_init.py::TestLazyEngineInit::test_lazy_engine_init_cpp_e2e - AssertionError: False is not true : Resnet18 TRT outputs don't match with the original model. Cosine sim score: nan Threshold: 0.99 FAILED runtime/test_002_lazy_engine_init.py::TestLazyEngineInit::test_lazy_engine_init_cpp_serialization -...
## Bug Description `FAILED runtime/test_003_cross_compile_for_windows.py::TestCrossCompileSaveForWindows::test_dynamo_cross_compile_for_windows_multiple_output - Failed: unexpected exception raised: Windows cross compilation feature is not available ` ## To Reproduce Steps to reproduce the behavior: 1. 2. 3. ##...
# Description Delete the extra copy INetworkDefinition creates Fixes # (issue) ## Type of change Please delete options that are not relevant and/or add your own. - Bug fix (non-breaking...
# Description Fixes # (issue) ## Type of change Please delete options that are not relevant and/or add your own. - Bug fix (non-breaking change which fixes an issue) -...
# Description This PR redesigns our LLM model compilation, unifies it, fixes output mismatch and performance issues. This PR also implements KV caching using native TensorRT. Fixes # (issue) ##...
