TensorRT
TensorRT copied to clipboard
pytorch
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
# Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change....
- There is no `getting_started` under `user_guide`. - `Tutorials` have its own section so remove duplicated index from `User Guide`. - `vgg16_fp8_ptq` was renamed to `vgg16_ptq`.
# Description Calling `convert_exported_program_to_serialized_trt_engine` with either `inputs=()` or `arg_inputs=()` is blocked by `AssertionError`. This is because the empty tuple is evaluated as `False` when it is implicitly casted to `bool`...
# Description Exported programs with dynamic shapes often include aten.constant_pad_nd with the argument `pad` containing mixture of `int` and `TRTTensor` values. This makes `pad_before + pad_after` fails in `get_padded_shape_tensors`. ##...
# Description - Weight streaming feature is exposed as compiler option to set percent or weight streaming bytes - Create a network with kSTRONGLY_TYPED and set kWEIGHT_STREAMING to builder config...
## Bug Description https://github.com/pytorch/TensorRT/blob/main/examples/dynamo/mutable_torchtrt_module_example.py I wasn't able to run this file above. I attempted to build the package from source but encountered an issue during the installation of torch version...
## Bug Description But it raises exception: RuntimeError: method.qualname() == QualifiedName(selfClass->name()->qualifiedName(), methodName)INTERNAL ASSERT FAILED at "../torch/csrc/jit/serialization/python_print.cpp":1105, please report a bug to PyTorch. ``` import torch.nn import torch_tensorrt class MySubmodule(torch.nn.Module): def...
## Bug Description The bug was observed while testing engine caching with `torch.compile(...)`. The error message is: ``` Traceback (most recent call last): File "/home/zewenl/anaconda3/envs/trt-10.1-py310/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1446, in _call_user_compiler compiled_fn...
# Description Engine caching feature. More details see: https://github.com/pytorch/TensorRT/discussions/2957 ## Type of change - New feature (non-breaking change which adds functionality) - This change requires a documentation update # Checklist:...
# Description Implement FP32 accumulation for matmul layers ## Type of change Please delete options that are not relevant and/or add your own. - Bug fix (non-breaking change which fixes...