Is it possible to support OPT models

[treya-lin]

Hi, instead of llama models, I wonder if this pipeline can be adjusted to fit OPT models? I wanna try OPT-1.3b model because llama 7b is still too big for my environment.

1. configuration-wise. in TencentPretrain/models, there is no config support for OPT models, may I ask how to get the config information for new models?

2. for this pipeline, if we use other large language models instead of llama-7b/13b model, what do we need to change to make it work?

Any suggestion is much appreciated!

[ydli-ai]

Thanks for your feedback, we are working on documentation to help you develop new models based on TencentPretrain.

It will be available soon.