TensorRT

TensorRT copied to clipboard

Add trt_executed_modules & default_torch_execution interface

2

Signed-off-by: Cheng Hang # Description According to the current design, every module is by default compiled to TensorRT execpt those modules included in torch_executed_modules. However, in some cases, for example,...

Njuapp

✨[Feature] Port notebooks to sphinx-gallery

2

**Is your feature request related to a problem? Please describe.** Notebooks right now have a few issues including: - Notebooks must be run AOT to avoid triggering execution during documentation...

narendasan

🐛 [Bug] Segmentation Fault When Trying to Quantize ResNet50 model

8

## Bug Description I'm using torch_tensorrt to try and quantize a pretrained ResNet50 model (roughly following the steps [here](https://nvidia.github.io/Torch-TensorRT/_notebooks/vgg-qat.html)), but I am getting a segmentation fault. I've tried running the...

henrycharlesworth

❓ [Question] Linking error with PTQ function

2

## ❓ Question I am getting a linking error when using `torch_tensorrt::ptq::make_int8_calibrator`. I am using the Windows build based on CMake, so I'm not sure if it's a problem with...

jonahclarsen

🐛 [Bug] None as const doesn't work

7

## Bug Description I tried to convert a scripted module to torch-tensorrt but it includes None as a const and it seems like that is not supported. ## To Reproduce...

naor2013

Support Loop Translation to TensorRT

12

In order to fully support RNNs, we want to be able to make recurrent subgraphs to TRT. To do so we must expand the capability of the compiler to recognize...

narendasan

🐛 [Bug] High accuracy decrase with torch f32 model and torch_tensorrt f32 model

1

## Bug Description L1 loss is too large between torch f32 and compiled torch_tensorrt model (base) root@VM-121-213-centos:/apdcephfs/share_1041553/kyikiwang/BasketDetect# python resnet_trt.py Using cache found in /root/.cache/torch/hub/pytorch_vision_v0.10.0 WARNING: [Torch-TensorRT] - Dilation not used...

kyikiwang

❓ [Question] Why BERT Base is slower w/ Torch-TensorRT than native PyTorch?

7

## ❓ Question I'm trying to optimize hugging face's BERT Base uncased model using Torch-TensorRT, the code works after disabling full compilation (`require_full_compilation=False`), and the avg latency is ~10ms on...

void-main

❓ [Question] Running a same torchscript using the same input producing different results.

4

## ❓ Question I'm trying to run a pretrained resnet50 model from torch.torchvision.models. enabled_precisions is set to torch.half. Each time I load the same resnet50 torchscript, using the same input（which...

SeTriones

🐛 [Bug] conv2d of sum results in "Expected orig_dims.nbDims > 2 to be true but got false. Unable to create convolution layer from node"

4

## To Reproduce ```python # https://github.com/bilibili/ailab/blob/main/Real-CUGAN/VapourSynth/upcunet_v3_vs.py import torch import torch_tensorrt from torch import nn as nn from torch.nn import functional as F import os,sys import numpy as np from tqdm...

styler00dollar