TensorRT
TensorRT copied to clipboard
pytorch
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
## Bug Description Error message: ```py torch._dynamo.exc.UserError: Cannot export model which references tensors that are neither buffers/parameters/constants nor are direct inputs. For each tensor, if you'd like this tensor to...
- Add converter or lowering pass for `torch.ops.aten.baddbmm.default`
Remove/replace: https://github.com/pytorch/TensorRT/blob/992ad70804a4967e60158095bd86c47bbe0ee1ae/py/torch_tensorrt/dynamo/lowering/passes/fuse_prims_broadcast.py#L22 Results in the following error on HF models: ```python File "/root/.pyenv/versions/3.10.14/lib/python3.10/site-packages/torch/_refs/__init__.py", line 3714, in _reshape_view_helper raise ValueError(msg) ValueError: Cannot view a tensor with shape torch.Size([256, 9, 32, 128])...
## Bug Description Currently, Torch-TRT displays the following error when compiling Llama-2-7B in FP16 on a 4090 GPU: ``` [05/08/2024-20:47:56] [TRT] [E] 1: [defaultAllocator.cpp::allocate::19] Error Code 1: Cuda Runtime (out...
# Description The `aten.atan2.out` operation calculates the element-wise arctangent of two tensors and stores the results in a specified output tensor, `out`. This does not alter the input tensors, meaning...
The converter for `aten.atan2.out ` The `aten.atan2.out` operation computes the element-wise arctangent of two tensors, storing the result in a pre-defined output tensor `out`, but does not modify the input...
# Description Dynamic shape cannot be used in where ops because of exception in torch.broadcast_shapes(). Proposed fix removes expand() for static shape input and only performs prepend ones to have...
# Description A converter for the torch.ops.aten.diagonal operation. There are very good reference for aten.diagonal operation ( https://github.com/pytorch/pytorch/blob/082251e76b93b277ff2791d0e2b64934add34644/torch/_refs/__init__.py#L4255 ) Fixes # ([issue](https://github.com/pytorch/TensorRT/issues/2846)) ## Type of change Please delete options that...
# Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change....
