Will OPT-IML-175B be supported?

[JingfengYang]

System information

• Alpa version: 0.2.2
• Are you willing to contribute it (Yes/No): No.

Describe the new feature and the current behavior/state Alpa supports OPT-175B currently. But there is a recent model of OPT-IML-175B, which is better than OPT released by Meta. Gthub link: https://github.com/facebookresearch/metaseq/tree/main/projects/OPT-IML .

Will this change the current API? How? No

Describe alternatives you've considered I have tried OPT-175B

Additional context

[jellyheadandrew]

+1

[zhisbug]

Any forms of contributions about OPT-IML are welcome

[GanjinZero]

+1

[skyz8421]

+1

[KazutoshiShinoda]

+1

[larekrow]

+1

[dlzou]

I investigated this and found that only minimal config changes are needed to support OPT-IML in the llm_serving example, since it has the same architecture as OPT. I added those changes in #894 and confirmed that textgen.py runs for up to OPT-IML-Max-30B on 8x Tesla P100s.

[larekrow]

I am looking for the conversion scripts/steps for the OPT-IML-175B weights, similar to the conversion for the OPT-175B weights. The metaseq OPT-175B weights are given as 992 FSDP shards while the metaseq OPT-IML-175B weights are given as 16 TPs, so I am not quite sure how I could run step 2.