TensorRT
TensorRT copied to clipboard
pytorch
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
## ❓ Question I am trying to follow the tutorial [here](https://pytorch.org/TensorRT/tutorials/serving_torch_tensorrt_with_triton.html) and am stuck at compiling the model with tensor-rt. The model i am using takes a list of tensors...
The majority of performance improvements related to SD + SDXL have focused on the UNet component. Expand the performance optimizations and focus to include the other components of the Stable...
See the failure in the `test-py-dynamo-core-cu121` [section here](https://github.com/pytorch/TensorRT/actions/runs/7350298746/job/20011958833?pr=2538). The hardware compatibility test fails when the TRT engine cannot be deserialized.
```[tasklist] ### Tasks - [ ] https://github.com/pytorch/TensorRT/issues/2581 - [ ] https://github.com/pytorch/TensorRT/issues/2582 - [ ] https://github.com/pytorch/TensorRT/issues/2583 ```
- Remove class override of `ConstantFolder` once Torch addresses the issue - Issue filed here: https://github.com/pytorch/pytorch/issues/110904 To remove: https://github.com/pytorch/TensorRT/blob/0e4c5d8617977768d21d346ea1b07d4abd0baa57/py/torch_tensorrt/dynamo/lowering/passes/constant_folding.py#L63-L71
## Bug Description Monai bug swin unet not working. Check nvbugs. Try dynamo ## Expected behavior ## Environment > Build information about Torch-TensorRT can be found by turning on debug...
## TL;DR ## Goal(s) ## Tasks ```[tasklist] ### Tasks ``` ## Additional context
## Bug Description I would expect for torch to raise an exception when inference fails for any reason, such as wrong input tensor shape or wrong dtype. Instead, a warning...
