TensorRT icon indicating copy to clipboard operation
TensorRT copied to clipboard

feat: Add validators for dynamic shapes in converter registration

Open peri044 opened this issue 1 year ago • 2 comments

Description

This PR does the following

  1. The default behavior is all converters are marked dynamic=False.
  2. We conservatively/explicitly mark dynamic=True for the nodes we know that support dynamic shapes.
  3. If a converter is marked dynamic=True, nothing happens. Both static and dynamic shapes are supported.
  4. If a converter is marked dynamic=False, it checks if the node received dynamic inputs. If it received dynamic inputs, we disable this converter and the node will fallback to PyTorch. If it received static inputs, compilation proceeds fine with no warnings.
  5. For static inputs to the converter, this flag doesn't affect anything.

Type of change

Please delete options that are not relevant and/or add your own.

  • New feature (non-breaking change which adds functionality)

Checklist:

  • [x] My code follows the style guidelines of this project (You can use the linters)
  • [x] I have performed a self-review of my own code
  • [x] I have commented my code, particularly in hard-to-understand areas and hacks
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have added tests to verify my fix or my feature
  • [ ] New and existing unit tests pass locally with my changes
  • [ ] I have added the relevant labels to my PR in so that relevant reviewers are notified

peri044 avatar Apr 29 '24 22:04 peri044

WARNING:torch_tensorrt.dynamo.conversion._ConverterRegistry:The converter for node aten.convolution.default received dynamic shaped inputs but the static version of the converter is being used. Please report this issue at https://github.com/pytorch/TensorRT/issues

This is a severe warning, it implies that it probably wont work. Is the intention for us to treat it as something that probably would work but might fail like (70/30) or like (20/80) pass/fail? If its the latter this should be an error, if its the former, we need to adjust the warning

narendasan avatar Apr 29 '24 23:04 narendasan

WARNING:torch_tensorrt.dynamo.conversion._ConverterRegistry:The converter for node aten.convolution.default received dynamic shaped inputs but the static version of the converter is being used. Please report this issue at https://github.com/pytorch/TensorRT/issues

This is a severe warning, it implies that it probably wont work. Is the intention for us to treat it as something that probably would work but might fail like (70/30) or like (20/80) pass/fail? If its the latter this should be an error, if its the former, we need to adjust the warning

Intention is "would work but might fail". How about the following message ?

The converter for node aten.convolution.default received dynamic shaped inputs although it was designed for static inputs. This shouldn't likely cause issues unless there are some dimensions which are dynamic (excluding the batch). If you encounter any issues, please post at https://github.com/pytorch/TensorRT/issues`

peri044 avatar Apr 30 '24 00:04 peri044