New Models: Starling Beta (7B) + Command-R (35B)

[JamesClarke7283]

I would like to see some new models added to hugging chat, i will provide my reasoning for each. One is a small model(7B), the other is larger (35B).

Starling Beta 7B

Link: HuggingFace Model

Introduction

Its a small model, but it seems to be on par with Mixtral on most benchmarks. its a finetune of openchat which is a finetune of mistral.

Reasoning

There are 3 reasons why i recommend this model:

1. Cheap to run (its only 7B parameters)
2. It outperforms Mixtral and GPT3.5 on the Arena Elo Leaderboard
3. Performance on par with Mixtral on most benchmarks on the Open LLM leaderboard.

Scores

Scores On Elo

1. Starling beta 7B: 1118
2. Mixtral: 1114
3. GPT3.5: 1103

Scores On OpenLLM Leaderboard

1. Starling beta 7B: 69.88%
2. Mixtral Instruct: 72.7%

Command-R

Link: HuggingFace Model

Command-R is a larger model (35B), It supports as long of a context length as GPT4-turbo and understands 10 languages, it also supports RAG and Tool Usage better than the other Open Source Models, including Mixtral, making the web search feature better with this model.

Reasoning

1. Longer 128k context.
2. Strong capabilities across 10 key languages(English, French, Spanish, Italian, German, Portuguese, Japanese, Korean, Arabic, and Chinese)
3. New model type, its good to use diverse models with different architectures.
4. Tool Usage and RAG: it is better optimized for tool usage, improving the web search capability in HuggingChat.

Scores

Scores on Elo

1. Command-R: 1146
2. Mixtral: 1114
3. GPT3.5: 1103

Scores on OpenLLM Leaderboard

1. Command-R: 68.54%
2. Starling beta 7B: 69.88%
3. Mixtral Instruct: 72.7%

Conclusion

In conclusion, adding the cost-effective and capable Starling Beta 7B model, along with the versatile and multi-lingual Command-R 35B model, would significantly enhance Hugging Chat's offerings. These additions would provide users with a more comprehensive AI experience and solidify Hugging Chat's position as a leading provider of cutting-edge AI technology.

[flexchar]

I'd actually be very curious about https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm :)

[JamesClarke7283]

I'd actually be very curious about https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm :)

Mainly I wanted to be conservative on cost. because dbrx is a expensive model to run(132B parameters), i would not think it would be added. Plus it does not have GGUF versions at all(yet) or GGUF quants yet on HF, if thats what HuggingChat uses.

Plus, that model is under a non-free license that prohibits further fine-tuning, so it might be a bit controversial to add it as the term "open" is a bit lose here.

In conclusion I would like to see it get added also, but if its too expensive i stand by the 7B and 35B models i recommended above.