TensorRT

TensorRT copied to clipboard

# Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change....

narendasan

[feat] Add dependency awareness to torch-trt partitioning

4

Adds a heuristic to torch-trt partitioning's segmentation to avoid materializing segments until we hit a dependency of that segment. This can significantly reduce the number of segments/engines in cases where...

mfeliz-cruise

How to correctly format input for Fp16 inference using torch-tensorrt C++

4

## ❓ Question ## What you have already tried Hi, I am using the following to export a torch scripted model to Fp16 tensorrt which will then be used in...

SM1991CODES

**Is your feature request related to a problem? Please describe.** Currently Torch-TensorRT is relying on Partitioning to support Tuple/List grouped inputs, which means for some models with Tuple/List input/output, they...

bowang007

Wrap ScriptFunctions in modules to compile

10

Add a section to `_compiler.py` that wraps ScriptFunctions in Modules to compile the graph.

narendasan

✨[Feature] torchtrtc should have the way to accept compile spec from command line

6

**Is your feature request related to a problem? Please describe.** I can't supply compile spec to trtorchc now. **Describe the solution you'd like** **Describe alternatives you've considered** **Additional context**

borisfom

✨[Feature] Look forward to a converter for torch.inverse

3

**Is your feature request related to a problem? Please describe.** A program can not be compiled with the "torch.inverse" operation.  **Describe the solution you'd like** Could you release this...

JuncFang-git

🐛 [Bug] Unsupported operator: aten::lstm.input, aten::Int.Tensor

5

## Bug Description ERROR: [Torch-TensorRT] - Method requested cannot be compiled by Torch-TensorRT.TorchScript. Unsupported operators listed below: - aten::lstm.input(Tensor input, Tensor[] hx, Tensor[] params, bool has_biases, int num_layers, float dropout,...

ilgrad

🐛 [Bug] nn.PReLU crashes torch_tensorrt.compile

5

## Bug Description Compiling model fails if `nn.PReLU` is used. It works fine with `nn.ReLU`. ## To Reproduce Just run this code ```python from torch import nn as nn from...

styler00dollar

✨[Feature] Bilinear Model Support

4

**Is your feature request related to a problem? Please describe.** Not related to a problem **Describe the solution you'd like** I have a model that has a Bilinear layer and...

agemagician