TensorRT
TensorRT copied to clipboard
pytorch
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
## Bug Description https://github.com/pytorch/TensorRT/blob/main/py/torch_tensorrt/dynamo/runtime/_PythonTorchTensorRTModule.py#L63-L65 This should be: ``` assert ( self.engine.num_io_tensors == len(self.input_names) + len(self.output_names) ) ``` because `num_io_tensors` is not inflated by the number of optimization profiles, for enqueueV3...
# Description Since CUDNN is no longer required by Torch-TRT, remove CUDNN dependencies. Fixes #2795 ## Type of change - This change requires a documentation update # Checklist: - [...
# Description This PR shows a simple example about using `accelerate` library for data parallel inference. # Checklist: - [ ] My code follows the style guidelines of this project...
**Is your feature request related to a problem? Please describe.** See https://github.com/pytorch/TensorRT/pull/2752 for an example. Some operators return a scalar, if these are on the graph boundary there may be...
# Description Cherry pick `embedding_bag` converter for release 2.3. Fixes # (issue) ## Type of change Please delete options that are not relevant and/or add your own. - Bug fix...
## Bug Description I am trying to speed up inference on huggingface Chinese BERT ([https://huggingface.co/bert-base-chinese](url)), and I need to do dynamic dimension inference acceleration.**But errors in the dynamic dimension confuse...
# Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change....
## Bug Description **Error Message:** 09T18:21:42.631Z INFO: pip is looking at multiple versions of torch-tensorrt to determine which version is compatible with other requirements. This could take a while. 2024-05-09T18:21:42.882Z...
# Description - Adds mirror functions and testing in C++ to control the `MULTI_DEVICE_SAFE_MODE` flag Fixes #2813 ## Type of change Please delete options that are not relevant and/or add...
# Description A converter for the torch.ops.aten.log1p operation (elementwise log(1 + x)) Fixes # ([issue](https://github.com/pytorch/TensorRT/issues/2760)) ## Type of change - New feature (non-breaking change which adds functionality) # Checklist: -...
