TensorRT
TensorRT copied to clipboard
pytorch
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
# Description This PR enables user settings to be accessed by our lowering system, which enables cleaner way of enabling/disabling passes (eg: use_fp32_acc). Fixes # (issue) ## Type of change...
## Bug Description it report _to_copy is not supported, when we use things like `x.to(torch.uint8)` ``` torch_tensorrt.dynamo.conversion._TRTInterpreter.UnsupportedOperatorException: Conversion of function torch._ops.aten.aten::_to_copy not currently supported! While executing %_to_copy : [num_users=1] =...
## Bug Description I am trying to use `torch_tensorrt.dynamo.compile()` to AOT compile the UNet portion of a `StableDiffusionPipeline` from the diffusers library (version 0.30.2). I am able to export the...
# Description Currently importing `torch_tensorrt` calls `_cuda_init`, which can fail, preventing torch_tensorrt being imported in environments without a (working) GPU. This makes it slow to import torch_tensorrt and isn't necessary....
``` 2024-08-31 09:14:02.849 | ERROR | MainProcess | /usr/local/lib/python3.10/dist-packages/torch_tensorrt/logging.py:24 - ITensor::getDimensions: Error Code 4: API Usage Error ([SCATTER]-[aten_ops.scatter.src]-[scatter_1_scatter_layer]: ScatterLayer in elements mode all inputs tensors rank must be same. Input...
**Is your feature request related to a problem? Please describe.** I am trying to compile our model with tensorrt these days with exported program, our model is not very big,...
## Bug Description when i use torch_tensorrt.compile transformer module with dynamic_shapes, it will occur this error, ## To Reproduce def test_compile_v1(): model = AutoModel.from_pretrained('bert-base-case', use_cache=False) # Enabled precision for TensorRT...
This is for scatter_reduce decomposition where include_self=False
# Description 1) Supported weight-stripped engine 2) Added REFIT_IDENTICAL flag Fixes #3146 ## Type of change - New feature (non-breaking change which adds functionality) # Checklist: - [ ] My...
The following lines- ``` if (isinstance(input_val, TRTTensor)) and ( input_val.dtype == trt.int8 or input_val.dtype == trt.int32 ): input_val = cast_trt_tensor(ctx, input_val, trt.float32, name) ``` are present in almost all the...