[WIP] [DO NOT REVIEW] Adding TRT support for HF T5

[HamidShojanazeri]

Description

Please include a summary of the feature or issue being fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #(issue)

Type of change

Please delete options that are not relevant.

• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [ ] This change requires a documentation update

Feature/Issue validation/testing

Please describe the tests [UT/IT] that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced. Please also list any relevant details for your test configuration.

• [ ] Test A

• [ ] Test B

• Logs

Checklist:

• [ ] Have you added tests that prove your fix is effective or that this feature works?
• [ ] Has code been commented, particularly in hard-to-understand areas?
• [ ] Have you made corresponding changes to the documentation?

[sagemaker-neo-ci-bot]

AWS CodeBuild CI Report

• CodeBuild project: torch-serve-build-win
• Commit ID: 5c036d60d1f981150ed0b2ad0cf61e3ee23629f2
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[sagemaker-neo-ci-bot]

AWS CodeBuild CI Report

• CodeBuild project: torch-serve-build-cpu
• Commit ID: 5c036d60d1f981150ed0b2ad0cf61e3ee23629f2
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[sagemaker-neo-ci-bot]

AWS CodeBuild CI Report

• CodeBuild project: torch-serve-build-gpu
• Commit ID: 5c036d60d1f981150ed0b2ad0cf61e3ee23629f2
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[msaroufim]

@HamidShojanazeri are you still interested in merging this PR?

[msaroufim]

This PR is relatively old, let's continue discussions on It in #1857

[codecov[bot]]

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@2b98375). Click here to learn what that means. The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #1557   +/-   ##
=========================================
  Coverage          ?   45.13%           
=========================================
  Files             ?       63           
  Lines             ?     2601           
  Branches          ?       60           
=========================================
  Hits              ?     1174           
  Misses            ?     1427           
  Partials          ?        0           

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more