nebuly icon indicating copy to clipboard operation
nebuly copied to clipboard

[Chatllama] Add support to other open-source models

Open diegofiori opened this issue 1 year ago • 0 comments

Description

LLaMA is a great model, however the weights license can be a huge barrier for users interested in the open-source LLM capabilities. The actual RLHF is designed around the LLaMA model, but it can be extended to other open-source models, like Flan-T5 or OPT.

TODO

  • [x] Extend support of ChatLLaMA to HuggingFace models
  • [ ] Add support for the encoder-decoder architectures (AutoModelsForSeq2Seq)
  • [x] Add support for LLaMA from HF.
  • [x] Add support for Cerebras GPT.
  • [ ] Test the support for
    • [ ] FlanT5
    • [x] GPTJ
    • [x] GPTNeoX
    • [x] OPT
    • [x] BLOOM
    • [x] BLOOMZ
    • [ ] Galactica
  • [ ] Implement unittest for the code

diegofiori avatar Mar 08 '23 13:03 diegofiori