TensorRT
TensorRT copied to clipboard
pytorch
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
# Description Random bool/integer type input for dynamic shape is casted from random float in [0,1) All zero input might cause false positive test result. Fixes # (issue) ## Type...
## Bug Description Configuration : `llm_examples_main` branch, current torch version : 2.4, transformers==4.41.2 One of the subgraphs is receiving a `SymInt` node (s0+1) which relies on other `SymInt`nodes. Hence the...
**Is your feature request related to a problem? Please describe.** Currently when we do a release, there are following manual steps to run on the local workstation: 1) 8 pip...
# Description Add an example of Phi 3 vision model using `torch.compile` backend. ## Type of change Please delete options that are not relevant and/or add your own. - New...
- Multi-device safe mode disabled + inference on multiple GPUs using Python threading demo script
## Bug Description ```bash WARNING:torch_tensorrt.dynamo.conversion.aten_ops_converters:Unable to import quantization op. Please install modelopt library (https://github.com/NVIDIA/TensorRT-Model-Optimizer?tab=readme-ov-file#installation) to add support for compiling quantized models INFO:torch_tensorrt.dynamo.utils:Using Default Torch-TRT Runtime (as requested by user) INFO:torch_tensorrt.dynamo.utils:Compilation...
## Bug Description 1) Should this be moved to aten_ops_converters.py? 2) I tried to pass in static shape ITensor end to the test case of this converter but got error:...
## Bug Description Using nightly, the CI produces this issue with cuda 12.4 ``` =========================== short test summary info ============================ FAILED models/test_models.py::TestModels::test_bert_base_uncased - RuntimeError: Schema not found for node. File...
## Bug Description The culprit seems to be https://github.com/pytorch/TensorRT/commit/65833002c58b4f02a37224bb516994f2c97faad3. ```bash holy@HOLYWU:~$ python3 test.py INFO:torch_tensorrt.dynamo.utils:Using Default Torch-TRT Runtime (as requested by user) INFO:torch_tensorrt.dynamo.utils:Compilation Settings: CompilationSettings(enabled_precisions={}, debug=True, workspace_size=0, min_block_size=1, torch_executed_ops=set(), pass_through_build_failures=False, max_aux_streams=None,...