TensorRT
TensorRT copied to clipboard
pytorch
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
# Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change....
# Description This PR does the following 1) The default behavior is all converters are marked `dynamic=False`. 2) We conservatively/explicitly mark `dynamic=True` for the nodes we know that support dynamic...
# Description Cherry-pick of: #2789 ## Type of change Please delete options that are not relevant and/or add your own. - New feature (non-breaking change which adds functionality) # Checklist:...
# Description - Cherry-pick of Windows builds to `main` (#2764) ## Type of change Please delete options that are not relevant and/or add your own. - Cherry-pick # Checklist: -...
🐛 [Bug] `torch.export.load` fails to load file saved with `torch_tensorrt.save` in PyTorch 2.4.0.dev
## Bug Description ``` DEBUG:torch_tensorrt.dynamo._compiler:Input graph: graph(): %linear_weight : [num_users=1] = get_attr[target=linear.weight] %linear_bias : [num_users=1] = get_attr[target=linear.bias] %x : [num_users=1] = placeholder[target=x] %view : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%x, [2,...
Can I convert the torchscript model to tensorrt format through torch_tensorrt? Is there any corresponding script that you can give me for reference?
## Bug Description ``` File "/home/dperi/.pyenv/versions/3.11.7/lib/python3.11/site-packages/torch/functional.py", line 126, in broadcast_shapes raise RuntimeError(f"Trying to create tensor with negative dimension ({shape[i]}): ({shape[i]})") RuntimeError: Trying to create tensor with negative dimension (-1): (-1)...
## ❓ Question ## What you have already tried I am trying to convert a transformer model to TRT in fp16 (fp32 works fine 🙂). It includes bunch of LayerNorms,...
# Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change....
# Description Readme update * Deprecation policy * Dynamo based instructions * Quickstart / quick installation * separated detailed installation + contribution guide
