TensorRT icon indicating copy to clipboard operation
TensorRT copied to clipboard

Add a "no_conversion" flow to torch-tensorrt

Open mfeliz-cruise opened this issue 3 years ago • 1 comments

Adds a "no_conversion" option to torch-tensorrt which when enabled will replace the standard conversion and engine insertion with an embedded function call for each convertible segment. This allows inspection of the partition without running conversion and the possibility to convert each engine individually in subsequent runs.

Future work: Allow this flow to run without a GPU to enable TRT convertibility/partitioning linting flows on host machines Partition without running shape propagation when in the no-convert flow

Fixes # (#1361)

Please delete options that are not relevant and/or add your own.

  • New feature (non-breaking change which adds functionality)

  • [ ] My code follows the style guidelines of this project (You can use the linters)

  • [ ] I have performed a self-review of my own code

  • [ ] I have commented my code, particularly in hard-to-understand areas and hacks

  • [ ] I have made corresponding changes to the documentation

  • [ ] I have added tests to verify my fix or my feature

  • [ ] New and existing unit tests pass locally with my changes

  • [ ] I have added the relevant labels to my PR in so that relevant reviewers are notified

mfeliz-cruise avatar Sep 16 '22 23:09 mfeliz-cruise

@bowang007 Could something like this solve our graph stitching issues? Say if the way partitioning works is it creates a bunch of methods but a method is either 100% PyTorch or 100% TRT. May make things like collections way easier too.

narendasan avatar Sep 19 '22 19:09 narendasan

This PR has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

github-actions[bot] avatar Dec 19 '22 00:12 github-actions[bot]