TensorRT

TensorRT copied to clipboard

❓ [Question] How to save the compiled while using torch.compile

1

For the example below, how do I save the compiled model? backend = "torch_tensorrt" tp_model = torch.compile( tp_model, backend=backend, options={ "truncate_long_and_double": True, "enabled_precisions": {torch.float32, torch.float16}, "use_python_runtime": True, "min_block_size": 1, },...

klin2024

Add support for JetPack 6.2 build

1

Add support for the jetpack6.2 build. Currently jetpack 6.2 has: cuda: 12.6 python: 3.10 tensorrt: 10.3 DLFW: 24.08 (pytorch: 2.6.0) Jetson now distribute wheels on https://pypi.jetson-ai-lab.dev/ jetpack6.2 wheels: https://pypi.jetson-ai-lab.dev/jp6/cu126

johnnynunez

❓ [Question] How wo you export a triton kernel with model to a serialized engine that can be run in c++?

10

## ❓ Question How wo you export a triton kernel with model to a serialized engine that can be run in c++? ## What you have already tried Read through...

cmgreen210

✨[Feature] Key configuration quick feedback workflows for PRs

1

**Is your feature request related to a problem? Please describe.** We need faster turnaround time on PRs being validated in CI. Right now builds take 20mins + 40min - 1.5hrs...

narendasan

 As you can see, `overview` word in the sidebar is bit misleading, it would help users if it was more descriptive.

deo-abhijit

🐛 [Bug] Encountered bug when using Torch-TensorRT

1

## Bug Description Presence of BUILD file in released wheel does break use of package in Bazel with rules-python (https://github.com/bazel-contrib/rules_python/issues/2780). ## To Reproduce Steps to reproduce the behavior: - See...

gfringeli

🐛 [Bug] Assertion error in test_cross_compile_for_windows

1

DLFW NGC container shows this error.

apbose

🐛 [Bug] quantized_resnet_test.py failed on no attribute 'EXPLICIT_PRECISION'

9

The example script fx/quantized_resnet_test.py in the Torch-TensorRT repository fails to execute due to the use of a deprecated attribute EXPLICIT_PRECISION in the TensorRT Python API. This attribute is no longer...

korkland

🐛 [Bug] Can't convert SSDLite320 MobilenetV3, Unsupported operator: aten::index.Tensor(Tensor self, Tensor?[] indices) -> (Tensor)

3

## Bug Description I trained a ssdlite320_320 mobilenetv3 large with Widerface datasets for face detection task. Here is what I received when running the `torch_tensorrt.compile()`: > (capstone) jetson@jetson-desktop:~/FaceRecognitionSystem/jetson/backend/python$ python test.py...

hungpham3112

✨[Feature] sampling within the model

3

Exporting a model that uses torch.Categorical().sample to sample from the logits. I currently have a (fixed length) loop within a torch.compile graph that includes sampling from the logits to choose...

jjh42