TensorRT
TensorRT copied to clipboard
NVIDIA
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
## Description I used C++tensorrt and found that the inference performance actually decreases in multi-threaded situations. For example, for a single inference of one image, the execution time of enqueue...
I'm trying to write a unit test for flash attention using version 0.14.0.dev2024100100. I noticed that `host_runtime_perf_knobs ` is a new feature in recent versions. Here are how I use...
Hi trt community; How ı can convert my depth-anyting-v2 model tensorrt its possible ? Ubuntu : 22:04 TensorRT: 10.4 Cuda : 12.2 GPU : RTX A2000
I'm using `onnxruntime==1.15.1` & `onnx=1.14.0` with `onnx-graphsurgeon==0.5.2`. I'm modifying the output of an `onnx` graph of an `XGBoost` model using the `Reshape` & `Graph` operators as follows respectively: ```python variable.name...
Fix 'clean_df' function in 'trex/df_preprocessing.py' module which misbehaves not selecting any existing columns to delete in order to cleanup dataframe for visualization. Resolves: #4161
## Description Problems with building cudla models using EngineCapability::kDLA_STANDALONE. We want to use patterns kDLA_STANDALONE to run the model, but we encounter the following error when compiling the model. the...
Simply try it yourself. Try to install the `tensorrt` Python package. The instructions do not work. The contents of `tensorrt_bindings`'s `__init__.py` doesn't make sense.
## Description fp16 conversion using polygraphy breaks model. for onnx fp16 conversion, the inference results is not exactly the same but close enough. it works as expected ## Environment **TensorRT...
in below official NVIDIA repo giving below error https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT @contentis @w-e-w Even though all requirements are installed this line of code gives below error utilities py import tensorrt as trt...
