It will be a good idea to support T5 series model.

[svjack]

Before llama come to our sight, Big Science release MT0 and T0, which are forerunners of llm, and at the end of 2022, some repos take T5-large and T5-3b T5-11b as their base to train their small llm. It seems that T5 series are more suitable to be the ability of Bert takes in this project. Are there any plans to support T5 ? 😊

[gitsailor5]

Hey @svjack! We've planned for T5 after a few pending releases, but if there's more interest, we can fast-track it!
( Just two more T5 requests on this thread, and we push it )