TensorRT
TensorRT copied to clipboard
pytorch
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
# Description Adds a uv lock file and some settings that allows users to use uv out of the box to trivially build the main branch with just `uv` +...
## ❓ Question im able to `torch.export` and generate an ExportedProgram with no issues for my model. upon compiling with `torch_tensorrt`... ```python ep = torch.export.load("...") example_inputs = ep.example_inputs[0] model =...
# Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change....
#2740 should be using this. Will change it once this PR is finalized
This validator is for the purpose of dynamic shape support
# Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change....
## Bug Description ## To Reproduce Steps to reproduce the behavior: 1. Run the code ```py model = models.resnet101(pretrained=False).eval().to("cuda") exp_program = torch.export.export(model, tuple(inputs)) enabled_precisions = {torch.float} debug = False workspace_size...
## ❓ Question I'm trying to run the `examples/dynamo/vgg16_fp8_ptq.y` example but got following error: ``` Traceback (most recent call last): File "/home/wh/generative_action/SynHSI/vgg_quat.py", line 232, in exp_program = torch.export.export(model, (input_tensor,)) File...
## ❓ Question I was following [the documentation](https://pytorch.org/TensorRT/user_guide/dynamic_shapes.html#dynamic-shapes) on compiling a model with dynamic input shape. When saving the compiled graph module (following [this](https://pytorch.org/TensorRT/user_guide/saving_models.html)), the new `torch_tensorrt.save(module, path, inputs)` API...
