TensorRT
TensorRT copied to clipboard
pytorch
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
## Bug Description python examples/apps/flux_demo.py --dtype fp16 --low_vram_mode --load_or_save save WARNING:torch_tensorrt.dynamo.runtime._MutableTorchTensorRTModule:Trying to move the original PyTorch model. This will cause CPU offloading failing and increase GPU memory usage.If this is...
## TL;DR Theres a number of docs we should be updating ## Goal(s) Reorient the documentation around the latest features. ## Tasks ```[tasklist] ### Tasks - Torch-TensorRT-RTX - Torch-TensorRT Debugger...
## Bug Description Timing cache is used during build time for storing autotuning results in TRT Enterprise. As TRT-RTX does not use autotuning, [@Hongyu Miao (HONGYUM)] removed the timing cache...
```py import torch import torch.nn as nn import torch_tensorrt as torchtrt class Model(nn.Module): def forward(self, x, y, z, a, b): x.index_add_(0, y, z) # x = x.index_add_(0, a, b) return...
## ❓ Question ## What you have already tried ## Environment > Build information about Torch-TensorRT can be found by turning on debug messages - PyTorch Version (e.g., 1.0): 2.8.0...
Hi Team, Thank you for your work on Torch-TensorRT. I would like to inquire if there are any plans to provide official support and pre-compiled C++ binaries (LibTorchTRT) for the...
Index_put converter has the following cases missing 1. Non consecutive indices ``` param( test_name="nonconsecutive_caseOne", source_tensor=torch.zeros([2,4,4,2], dtype=torch.float32), indices_tensor=(None, torch.tensor([0, 0, 1, 1], dtype = torch.int64), None, torch.tensor([0, 0, 1, 1], dtype=torch.int64)),...
## Bug Description Serialization of Graph Break Fails ## To Reproduce Steps to reproduce the behavior: ``` import torch import torch.nn as nn import torch_tensorrt as torchtrt import torchvision torch.manual_seed(0)...