TensorRT

TensorRT copied to clipboard

🐛 [Bug] Part of the weights are placed to CPU during compilation

3

## Bug Description When compiling Bert, a device mismatch occurs. This seems to be caused by weights moved to CPU during compilation. ## To Reproduce Steps to reproduce the behavior:...

cehongwang

ValueError: Invalid input type <class 'bool'> encountered when compiling FLUX.1-dev model with Torch-TensorRT

8

## ❓ Question When trying to compile the FLUX.1-dev model using Torch-TensorRT following the official example/blog post, I'm encountering a `ValueError` during the `torch_tensorrt.dynamo.compile()` step. The error suggests there's an...

yachty66

**Is your feature request related to a problem? Please describe.** Support FP4 functionality on Blackwell using the ModelOpt workflow initially **Describe the solution you'd like** https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/7eecd11b0fde7a589b5568f17e19fb67c665730d/tests/torch/gpu/quantization/test_torch_export.py#L103 Support networks created with...

narendasan

output mismatch in DLFW NGC container

apbose

## Bug Description error when compile an LSTM model using torch tensorrt ## To Reproduce ``` import os import torch import torch.nn as nn import torch.nn.functional as F import torch_tensorrt...

zmy1116

❓ [Question] How do you properly deploy a quantized model with tensorrt

6

## ❓ Question I have a PTQ model and a QAT model trained with the official pytorch API following the quantization tutorial, and I wish to deploy them on TensorRT...

Urania880519

Installing TensorRT on Jetson Nano

1

Hi, Sorry for a noobish question, I'm just looking for a way how to install TernsorRT to Jetson Nano. So far, I'm using Ubuntu 20.04 from Q-Engineering with Pytorch 1.13...

aiotko

Tensor parallel Llama3 tutorial illustrating use of torch.distributed and nccl ops # Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation...

apbose

## Bug Description https://github.com/pytorch/TensorRT/actions/runs/13168805591/job/36755789477?pr=3382 ## To Reproduce Steps to reproduce the behavior: 1. 2. 3. ## Expected behavior ## Environment > Build information about Torch-TensorRT can be found by turning...

peri044

## [DEBUG | torch_tensorrt.dynamo.partitioning._global_partitioner]: Unsupported or Excluded Nodes: - torch.ops.aten.native_dropout.default + Operator Count: 2 - **Function Schema**: - **Original PyTorch API**: - **Relevant TensorRT Documentation**: ## Alternatives ## Additional context

peri044