TensorRT
TensorRT copied to clipboard
🐛 [Bug] arange converter dynamic shape bug
Bug Description
- Should this be moved to aten_ops_converters.py?
- I tried to pass in static shape ITensor end to the test case of this converter but got error: aten::arange() Expected a value of type 'number' for argument 'end' but instead found type 'FakeTensor'. because the schema says the type of the arguments is Scalar
- In terms of the implementation, 1) shape seems not to be always valid, 2) output type is decided by end.dtype but it's possible the start is float and end is int. e.g.
torch.arange(1.2, 15, 2.2) tensor([ 1.2000, 3.4000, 5.6000, 7.8000, 10.0000, 12.2000, 14.4000]) torch.ops.aten.div.Tensor_mode(13.8, 2.2, rounding_mode="trunc") tensor(6.) I think the actual length should be 7 but the length of shape is 6. Please correct me if I'm wrong.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Environment
Build information about Torch-TensorRT can be found by turning on debug messages
- Torch-TensorRT Version (e.g. 1.0.0):
- PyTorch Version (e.g. 1.0):
- CPU Architecture:
- OS (e.g., Linux):
- How you installed PyTorch (
conda,pip,libtorch, source): - Build command you used (if compiling from source):
- Are you using local sources or building from archives:
- Python version:
- CUDA version:
- GPU models and configuration:
- Any other relevant information:
Additional context
@zewenli98
- Having it in op_evaluators is fine due to its duality
- How are you passing static shape ITensor to the converter ? Can you show your testcase ?
- Seems like a bug. I'll work on a fix.
- I used test case like:
def test_arange_dynamic(self):
class Arange(nn.Module):
def forward(self, end_tensor):
return torch.ops.aten.arange.start_step(0, end_tensor, 1)
pyt_input = 7
inputs = [
torch.tensor(pyt_input, dtype=torch.int32),
]
self.run_test(
Arange(),
inputs,
)
and got the error:
...
arange_start_step = torch.ops.aten.arange.start_step(0, end_tensor, 1); end_tensor = None
File "/home/zewenl/anaconda3/envs/trt-10-py310/lib/python3.10/site-packages/torch/_ops.py", line 610, in __call__
return self_._op(*args, **kwargs)
RuntimeError: aten::arange() Expected a value of type 'number' for argument 'end' but instead found type 'Tensor'.
Position: 1
Value: tensor(7, device='cuda:0', dtype=torch.int32)
Declaration: aten::arange.start_step(Scalar start, Scalar end, Scalar step=1, *, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=None) -> Tensor
Cast error details: Cannot cast tensor(7, device='cuda:0', dtype=torch.int32) to number
To execute this test, run the following from the base repo dir:
python test_arange_aten.py -k test_arange_dynamic
This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
----------------------------------------------------------------------
Ran 1 test in 0.149s
The schema shows arange and its variants only support Scalar instead of Tensor