TensorRT

TensorRT copied to clipboard

fix: replace add_identity by add_cast for type cast

5

# Description This PR updates the `type_cast` helper function to ensure compatibility with TensorRT's strongly typed network mode. `type_cast` used `add_identity()` followed by `set_output_type()` to perform the data type cast....

junstar92

## Bug Description ``` FAILED runtime/test_004_weight_streaming.py::TestWeightStreamingPython::test_weight_streaming_default_0_python_runtime - AssertionError: Tensor-likes are not close! FAILED runtime/test_004_weight_streaming.py::TestWeightStreamingPython::test_weight_streaming_default_1_cpp_runtime - AssertionError: Tensor-likes are not close! FAILED runtime/test_004_weight_streaming.py::TestWeightStreamingPython::test_weight_streaming_manual_0_python_runtime - AssertionError: Tensor-likes are not close! FAILED runtime/test_004_weight_streaming.py::TestWeightStreamingPython::test_weight_streaming_manual_1_cpp_runtime...

narendasan

## Bug Description ``` __________________________________________________________________________________________________________________________________________________ TestAutomaticPlugin.test_mul_plugin_float_0 __________________________________________________________________________________________________________________________________________________ a = (,), kw = {} @wraps(func) def standalone_func(*a, **kw): > return func(*(a + p.args), **p.kwargs, **kw) ../.venv/lib/python3.9/site-packages/parameterized/parameterized.py:620: _ _ _ _ _...

narendasan

🐛 [Bug] Test failure `tests.py.ts.api.test_classes`

1

## Bug Description ``` api/test_classes.py:6: in import torch_tensorrt as torchtrt /opt/python/cp39-cp39/lib/python3.9/site-packages/torch_tensorrt/__init__.py:125: in from torch_tensorrt.runtime import * # noqa: F403 /opt/python/cp39-cp39/lib/python3.9/site-packages/torch_tensorrt/runtime/__init__.py:1: in from torch_tensorrt.dynamo.runtime import ( # noqa: F401 /opt/python/cp39-cp39/lib/python3.9/site-packages/torch_tensorrt/dynamo/__init__.py:10: in...

narendasan

## Bug Description ``` api/test_classes.py:6: in import torch_tensorrt as torchtrt /opt/python/cp39-cp39/lib/python3.9/site-packages/torch_tensorrt/__init__.py:125: in from torch_tensorrt.runtime import * # noqa: F403 /opt/python/cp39-cp39/lib/python3.9/site-packages/torch_tensorrt/runtime/__init__.py:5: in from torch_tensorrt.runtime._cudagraphs import ( /opt/python/cp39-cp39/lib/python3.9/site-packages/torch_tensorrt/runtime/_cudagraphs.py:61: in class _CudagraphsContextManager(object): /opt/python/cp39-cp39/lib/python3.9/site-packages/torch_tensorrt/runtime/_cudagraphs.py:74:...

narendasan

TRT-LLM download utility

apbose

slice scatter support for dynamic cases

1

This is with reference to #3448. The error currently being faced is that- (div_2, sym_size_int_3669, mul_102, mul_114, reshape_default_3, mul_213, div_113, mul_2962, clone_54, select_1, clone_67, expand_122, slice_16, clone_68, expand_123, mul_4869, expand_184,...

apbose

🐛 [Bug] AssertionError: end must be an integer

22

## Bug Description ## To Reproduce Steps to reproduce the behavior: 1. Prepare torch-tensorrt convert code ``` model = MyModel(opt) model.load_staste_dict(torch.load("model.pth")) inputs = torch_tensorrt.Input(min_shape=[int(min_batch_size), channel, height, width], opt_shape=[int(max_batch_size // 2),...

Fulitcher

# Description This PR includes the AOT Plugin demo for RMSNorm Triton kernel. # Checklist: - [ ] My code follows the style guidelines of this project (You can use...

bowang007

# Description Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change....

lanluo-nvidia