TensorRT
TensorRT copied to clipboard
pytorch
🐛 [gpt2][export] Constraint violation error during GPT2 export
Bug Description
Configuration : llm_examples_main branch, current torch version : 2.4, transformers==4.41.2 Error message:
File "/home/dperi/Downloads/TensorRT/examples/dynamo/torch_export_gpt2.py", line 64, in <module>
gpt2_ep = export_gpt2(model, input_ids)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dperi/Downloads/TensorRT/examples/dynamo/torch_export_gpt2.py", line 19, in export_gpt2
ep = torch.export.export(
^^^^^^^^^^^^^^^^^^^^
File "/home/dperi/.pyenv/versions/3.11.7/lib/python3.11/site-packages/torch/export/__init__.py", line 174, in export
return _export(
^^^^^^^^
File "/home/dperi/.pyenv/versions/3.11.7/lib/python3.11/site-packages/torch/export/_trace.py", line 945, in wrapper
raise e
File "/home/dperi/.pyenv/versions/3.11.7/lib/python3.11/site-packages/torch/export/_trace.py", line 928, in wrapper
ep = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/dperi/.pyenv/versions/3.11.7/lib/python3.11/site-packages/torch/export/exported_program.py", line 89, in wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/dperi/.pyenv/versions/3.11.7/lib/python3.11/site-packages/torch/export/_trace.py", line 1455, in _export
aten_export_artifact = export_func(
^^^^^^^^^^^^
File "/home/dperi/.pyenv/versions/3.11.7/lib/python3.11/site-packages/torch/export/_trace.py", line 1344, in _non_strict_export
raise UserError(UserErrorType.CONSTRAINT_VIOLATION, str(e)) # noqa: B904
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch._dynamo.exc.UserError: Constraints violated (seq_len)! For more information, run with TORCH_LOGS="+dynamic".
- Not all values of seq_len = L['args'][0][0].size()[1] in the specified range seq_len <= 1024 satisfy the generated guard Ne(1024, L['args'][0][0].size()[1]).
- Not all values of seq_len = L['args'][0][0].size()[1] in the specified range seq_len <= 1024 satisfy the generated guard Ne(L['args'][0][0].size()[1], 1024).
To Reproduce
Steps to reproduce the behavior:
python examples/dynamo/torch_export_gpt2.py
Expected behavior
Environment
Build information about Torch-TensorRT can be found by turning on debug messages
- Torch-TensorRT Version (e.g. 1.0.0):
- PyTorch Version (e.g. 1.0):
- CPU Architecture:
- OS (e.g., Linux):
- How you installed PyTorch (
conda,pip,libtorch, source): - Build command you used (if compiling from source):
- Are you using local sources or building from archives:
- Python version:
- CUDA version:
- GPU models and configuration:
- Any other relevant information:
Additional context
The solution is to not use max=1024. seq_len = torch.export.Dim("seq_len", min=1, max=1023) works