Port Huggingface T5v1_1 weights to NeMo-Megatron

[MaximumEntropy]

What does this PR do ?

Adds a conversion script and related compatibility args to port Huggingface T5v1_1 weights to NeMo-Megatron.

Collection: NLP

Changelog

• Adds a state dict conversation script.
• Adds necessary args to yaml configs to make sure the forward passes in HF and NeMo result in identical results.

Usage

• You can potentially add a usage example below

python scripts/nlp_language_modeling/hf_t5_to_nemo_coverter.py \
    --hf_model_name google/ul2 \
    --nemo_state_dict /path/to/nemo_ul2_state_dict.pt

Before your PR is "Ready for review"

Pre checks:

• [ ] Make sure you read and followed Contributor guidelines
• [ ] Did you write any new necessary tests?
• [ ] Did you add or update any necessary documentation?
• [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • [ ] Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

• [ ] New Feature
• [ ] Bugfix
• [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed. Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

• Related to # (issue)

[lgtm-com[bot]]

This pull request introduces 4 alerts when merging 88fb0a5a17969534473119e8039477c302d79a51 into 987674e29ea90f9a2f663bf95d74bd947d76bbc0 - view on LGTM.com

new alerts:

• 3 for Wrong name for an argument in a class instantiation
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 4 alerts when merging 984621d282515a5976ec89621a58264aa98d9e24 into 987674e29ea90f9a2f663bf95d74bd947d76bbc0 - view on LGTM.com

new alerts:

• 3 for Wrong name for an argument in a class instantiation
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 4 alerts when merging 69e5865155640b95fc7db3c6c239e3ec754a5977 into f921ebe0436e55f7547b183ca83a623f6678422d - view on LGTM.com

new alerts:

• 3 for Wrong name for an argument in a class instantiation
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 4 alerts when merging 479aa669cf4537984f79b2a8a4d40ed20b9c14c5 into f921ebe0436e55f7547b183ca83a623f6678422d - view on LGTM.com

new alerts:

• 3 for Wrong name for an argument in a class instantiation
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 4 alerts when merging 73d3cc1b9cc811667646f3a3bc1fd235ea42bb27 into 66c7677cd4a68d78965d4905dd1febbf5385dff3 - view on LGTM.com

new alerts:

• 3 for Wrong name for an argument in a class instantiation
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 4 alerts when merging 5c2bc6da585228d8b4bd4a03382d8cf413172298 into 66c7677cd4a68d78965d4905dd1febbf5385dff3 - view on LGTM.com

new alerts:

• 3 for Wrong name for an argument in a class instantiation
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 5 alerts when merging 8b7bcb56700f86ed7e97fcee22589670a28c1237 into 66c7677cd4a68d78965d4905dd1febbf5385dff3 - view on LGTM.com

new alerts:

• 3 for Wrong name for an argument in a class instantiation
• 1 for Unused import
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 5 alerts when merging 156e95a643f5099883ee7a39f904d2b0c0a64aef into 66c7677cd4a68d78965d4905dd1febbf5385dff3 - view on LGTM.com

new alerts:

• 3 for Wrong name for an argument in a class instantiation
• 1 for Unused import
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 3 alerts when merging bfa1a1a4f135f2efac479ed78ba2255d3a4cb477 into 4cd9b3449cbfedc671348fbabbe8e3a55fbd659d - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times
• 1 for Wrong name for an argument in a class instantiation

[lgtm-com[bot]]

This pull request introduces 3 alerts when merging 51829026d4cbeabaa7c15249d751681137b5e34d into 4cd9b3449cbfedc671348fbabbe8e3a55fbd659d - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times
• 1 for Wrong name for an argument in a class instantiation

[lgtm-com[bot]]

This pull request introduces 3 alerts when merging 48c603ff218e6d651d07bdffc236e9e95be7f75c into 4cd9b3449cbfedc671348fbabbe8e3a55fbd659d - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times
• 1 for Wrong name for an argument in a class instantiation

[lgtm-com[bot]]

This pull request introduces 3 alerts when merging e2ddae3439b755fc69fe601e497fe5a7f8f604d2 into 4cd9b3449cbfedc671348fbabbe8e3a55fbd659d - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times
• 1 for Wrong name for an argument in a class instantiation

[lgtm-com[bot]]

This pull request introduces 3 alerts when merging 0886fb9148bfb5af2904ffdf7eaab25574a98479 into da9f4138137565fe048a3f99dac343dcfd40aee4 - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times
• 1 for Wrong name for an argument in a class instantiation

[lgtm-com[bot]]

This pull request introduces 2 alerts when merging f12bbcb4355f338a9a2ac7c87d8c3cb4350afd06 into f8ca550967a83473aa2c20267690ac59c4fb640f - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 2 alerts when merging d459996b867e915de44107df36ae1fd669a9be20 into f8ca550967a83473aa2c20267690ac59c4fb640f - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 2 alerts when merging ea8aa507cf8ac0474f07d4706dbd034b7b1a6c67 into f8ca550967a83473aa2c20267690ac59c4fb640f - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 2 alerts when merging fbb1aa8caf2646e306a7e3b5f30e424789fcfc65 into f8ca550967a83473aa2c20267690ac59c4fb640f - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 2 alerts when merging 035fcf39dbbdbcc743799cc0d477f412d4cf27c6 into f8ca550967a83473aa2c20267690ac59c4fb640f - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 2 alerts when merging 681bb8d35530519af94a55fead0c29100ceb1fd4 into f8ca550967a83473aa2c20267690ac59c4fb640f - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 2 alerts when merging 4c7c385ae493b5ed16f5b8901ee0ca595ab010f5 into f7b7985ea8b42c958c133a6366fc69aaed9ec7c9 - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 2 alerts when merging 19806ad88b4a681009a30e7633dda072076fb362 into 193a0c3216f14ce6e9ca6dd4934b14788a63cb64 - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 2 alerts when merging 19806ad88b4a681009a30e7633dda072076fb362 into 193a0c3216f14ce6e9ca6dd4934b14788a63cb64 - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 2 alerts when merging 62d49bd1d34a551a04b83e278a5ad0f555308796 into 06ad865454e8007836659d26a99f4ce4c2e2905c - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times

[lgtm-com[bot]]

This pull request introduces 2 alerts when merging 2cc549bb1ee75d2c69ee1617df3bb6583173abf6 into 7357d4b107c821ff9f890ae1b9d9d0b9fe207890 - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times

[github-actions[bot]]

This PR is stale because it has been open for 30 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.