TensorRT
TensorRT copied to clipboard
feat: Add validators for dynamic shapes in converter registration
Description
This PR does the following
- The default behavior is all converters are marked
dynamic=False. - We conservatively/explicitly mark
dynamic=Truefor the nodes we know that support dynamic shapes. - If a converter is marked dynamic=True, nothing happens. Both static and dynamic shapes are supported.
- If a converter is marked dynamic=False, it checks if the node received dynamic inputs. If it received dynamic inputs, we disable this converter and the node will fallback to PyTorch. If it received static inputs, compilation proceeds fine with no warnings.
- For static inputs to the converter, this flag doesn't affect anything.
Type of change
Please delete options that are not relevant and/or add your own.
- New feature (non-breaking change which adds functionality)
Checklist:
- [x] My code follows the style guidelines of this project (You can use the linters)
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas and hacks
- [ ] I have made corresponding changes to the documentation
- [ ] I have added tests to verify my fix or my feature
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have added the relevant labels to my PR in so that relevant reviewers are notified
WARNING:torch_tensorrt.dynamo.conversion._ConverterRegistry:The converter for node aten.convolution.default received dynamic shaped inputs but the static version of the converter is being used. Please report this issue at https://github.com/pytorch/TensorRT/issues
This is a severe warning, it implies that it probably wont work. Is the intention for us to treat it as something that probably would work but might fail like (70/30) or like (20/80) pass/fail? If its the latter this should be an error, if its the former, we need to adjust the warning
WARNING:torch_tensorrt.dynamo.conversion._ConverterRegistry:The converter for node aten.convolution.default received dynamic shaped inputs but the static version of the converter is being used. Please report this issue at https://github.com/pytorch/TensorRT/issuesThis is a severe warning, it implies that it probably wont work. Is the intention for us to treat it as something that probably would work but might fail like (70/30) or like (20/80) pass/fail? If its the latter this should be an error, if its the former, we need to adjust the warning
Intention is "would work but might fail". How about the following message ?
The converter for node aten.convolution.default received dynamic shaped inputs although it was designed for static inputs. This shouldn't likely cause issues unless there are some dimensions which are dynamic (excluding the batch). If you encounter any issues, please post at https://github.com/pytorch/TensorRT/issues`