TensorRT

TensorRT copied to clipboard

❓ [Question] Is there any way to deploy on a single machine with multi-gpus？

1

## ❓ Question ## What you have already tried ## Environment > Build information about Torch-TensorRT can be found by turning on debug messages - PyTorch Version (e.g., 1.0): -...

SZ-ing

🐛 [Bug] AttributeError: module \'torch_tensorrt\' has no attribute \'set_device\'

1

## Bug Description AttributeError: module \'torch_tensorrt\' has no attribute \'set_device\' ## To Reproduce ``` >>> import torch_tensorrt >>> torch_tensorrt.set_device(0) Traceback (most recent call last): File "", line 1, in AttributeError:...

KumoLiu

🐛 [Bug] select scatter decomposition device handling

1

## Bug Description The input and src tensor can be on cpu but the index tensor is being casted to gpu:0 and this results in src and index tensor are...

peri044

🐛 [Bug] SDv1.5 generate black image with torch_tensorrt compile

3

## Bug Description SDv1.5 generate black image with torch_tensorrt compile log on cmd: Lib\site-packages\diffusers\image_processor.py:97: RuntimeWarning: invalid value encountered in cast images = (images * 255).round().astype("uint8") Potential NSFW content was detected...

xduzhangjiayu

❓ [Question] function `torch._ops.aten.aten::_to_copy` not currently supported with dynamic input shape

1

## ❓ Question I'm trying to compile a model with dynamic input shape but told that the `function torch._ops.aten.aten::_to_copy` is not currently supported: ```Traceback (most recent call last): File "/home/wh/generative_action/SynHSI/test_module.py",...

broken-dream

🐛 [Bug] Encountered bug : torch._dynamo.exc.InternalTorchDynamoError: 'NoneType' object is not subscriptable

1

## Bug Description I met a bug when I use torch-tensorrt to quantize a YOLOv6 model into fp8_e4m3. I use the code in [vgg_fp8_code](https://github.com/pytorch/TensorRT/blob/main/examples/dynamo/vgg16_fp8_ptq.py) , but I met the bug...

yyChen233

# Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change....

cehongwang

🐛 [Bug] ValueError: Trying to flatten user inputs with exported input tree spec

2

## Bug Description ## To Reproduce The code comes from the official documentation： https://pytorch.org/TensorRT/user_guide/dynamic_shapes.html#custom-dynamic-shape-constraints ```python import torch import torch_tensorrt class MatMul(torch.nn.Module): def __init__(self): super().__init__() def forward(self, query, key): attn_weight =...

Hukongtao

❓ [Question] How do you save a unet model compiled Torch-TensorRT (Stable Diffusion XL)

2

## ❓ Question How do you save a unet model compiled Torch-TensorRT from Stable Diffusion XL? ## What you have already tried I've tried following the compilation instructions from the...

dru10

🐛 [Bug] Encountered bug when using Torch-TensorRT - SD1.5 compilation causes incorrect output

2

## Bug Description Stable Diffusion 1.5 fails to be compiled correctly using dynamo.compile. In early June (when I opened refitter-support branch), SD 1.5 could compile correctly, but after I did...

cehongwang