TensorRT
TensorRT copied to clipboard
NVIDIA
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
## Description ## Environment **TensorRT Version**: 8.4 **NVIDIA GPU**: NVIDIA GeForce RTX 3060 Laptop GPU **NVIDIA Driver Version**: Driver Version: 515.65.01 **CUDA Version**: Using container **CUDNN Version**: Using Container **Operating...
## Description Gathering topk index on a **multi-batch** tensor gives unexpected results. Note that if we replace the profile with: ```python C=10 input_shapes = { 'input': { 'min_shape': [1, C,...
## Description When I use the TensorRT C++ API and Layernorm plugin to build my network, it's segmentation fault! I debug the Layernorm.cpp, it's wrong in this code: https://github.com/NVIDIA/TensorRT/blob/87f3394404ff9f9ec92c906cd4c39b5562aea42e/plugin/skipLayerNormPlugin/skipLayerNormPlugin.cpp#L430 and...
Hi, In reference to [#10807](https://github.com/microsoft/onnxruntime/issues/10807), I'm trying to use my own T5-base on TensorRT. My model has a `max_length=1024`, and using TensorRT (I created the TensorRT container) I have worse...
## Description I have a model that uses a slice operator for feature crossing, but it turns out that the slice operator calls the CopyPackedKernel API, and it consumes a...
Hi team, I am interested to see the CUDA source code of these transformer kernels, like: https://github.com/NVIDIA/TensorRT/blob/e5f9ead4a4826cc774325720a26dbf4ec47203ea/plugin/bertQKVToContextPlugin/fused_multihead_attention/src/fused_multihead_attention_fp16_128_64_kernel.sm75.cpp Anyone know where to see the source code?
Difference between firstTranspose and SecondTranspose？
Torch-TensorRT acts as an extension to TorchScript. But there are models that are not supported by pytorch jit tracing, see [warning in jit tracing doc](https://pytorch.org/docs/stable/generated/torch.jit.trace.html). Would you guys have plan...
hi there I have installed jetpack4.4 and everything works fine i juts need to install tensorrt7 manually as SDK is not working with me!!! please could you provide the link...
Hello, Could you please help me here, it seems that they abanded me. Please have a look below : https://forums.developer.nvidia.com/t/tensorrt-int8-calibration-python-api/227297 Thank you in advance. Best regards, Harry
