TensorRT
TensorRT copied to clipboard
pytorch
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
- Get CI up and running for Windows implementation using only the Python runtime
Hi Torch-TensorRT team: I'm wondering how can I compile a model with 8 bit weights, but using 16 bit activations? Thanks a lot!
## Bug Description RuntimeError: [Error thrown at core/partitioning/shape_analysis.cpp:183] Expected ivalues_maps.count(input) to be true but got false Could not find torch::jit::Value* hidden.1 produced from %hidden.1 : (Tensor, Tensor) = prim::TupleConstruct(%440, %440)...
## TL;DR Prioritze coverage for the [core ATen opset](https://pytorch.org/docs/main/torch.compiler_ir.html#core-aten-ir). ## Goal(s) - Determine priority order, based on criteria such as key model requirements, which operators from the core ATen opset...
# Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change....
# Description - Update `setup.py` to enable `bdist_wheel` build - Add GHA tooling for Windows Fixes #2489 ## Type of change - New CI Path # Checklist: - [ ]...
- Explicit considerations of truncation of inputs versus truncation of constants Sourced from https://github.com/pytorch/TensorRT/pull/2457#issuecomment-1889824984: - Avoid running PyTorch graphs with invalid casts - Refactor `repair_long_and_double` to consume output of type...
## ❓ Question I am within the nvcr.io/nvidia/pytorch:23.12-py3 container. The performance of torch_tensorrt is wrose than inductor. Details: example code ```python import torch import torch_tensorrt import torch.nn as nn class...
