Add OpenHermes or DPO variants NeuralHermes

[KeithCu]

Feature request

Mistral-OpenHermes seems to be overall better than the other Mistral fine-tunes:

https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B

Can you consider to offer this too?

Also, this is a pretty good program already, it would be cool if it could have sponsorship for more paid-contributors. There are so many cool features this program could have.

Motivation

The HuggingFace link above goes into some detail on the results.

Your contribution

I could submit a PR, but I'd have to do research ;-)

[KeithCu]

BTW, I did find on this link on some of the likely datasets it was trained on: https://huggingface.co/teknium/OpenHermes-13B

That was back when it was just 242K entries:

GPTeacher - General Instruct, Roleplay v1, Roleplay v2, and Code Instruct Datasets, by Teknium
WizardLM (v1, evol_instruct 70k), by WizardLM Team/nlpxucan
Airoboros GPT-4 (v1.0), by JonDurbin
Camel-AI's domain expert datasets, by the Camel-AI Team
CodeAlpaca, by Sahil2801
GPT4-LLM and Unnatural Instructions, by Microsoft

[kashw1n]

hi guys - i've raised a PR re: this https://github.com/nomic-ai/gpt4all/pull/1689

[KeithCu]

Since then, various DPO version further fine-tuned on OpenHermes 2.5 seem to have further improved the scores everywhere, especially the GSM8K.

So actually, consider offering this instead or perhaps as well: https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B

I could file a separate bug but there are plenty already so I renamed the title to be about the variants. I've not played with the NeuralHermes yet to compare whether it's truly better than OpenHermes which is already awesome, but I just wanted to put in this comment for now.

[KeithCu]

Okay, I've played with NeuralHermes on LMStudio for an hour and it seems even better than OpenHermes 2.5. My suggestion is to just use NeuralHermes instead.

[KeithCu]

BTW, I found out that NeuralHermes was a pretty basic DPO, and so this model below seems to be very interesting merge of OpenHermes 2.5 with Intel's Neural chat which is a DPO fine-tune.

https://huggingface.co/TheBloke/OpenHermes-2.5-neural-chat-v3-3-Slerp-GGUF

I've played with it and it seems very solid, so I'd consider this also. OpenHermes is amazing, so please put this variant of it in GPT4all soon. I could do the pull request if you want. I have a bit of time now.

[KeithCu]

BTW, I found out that NeuralHermes was a pretty basic DPO, and so this model below seems to be very interesting merge of OpenHermes 2.5 with Intel's Neural chat which is a DPO fine-tune.

https://huggingface.co/TheBloke/OpenHermes-2.5-neural-chat-v3-3-Slerp-GGUF

I've played with it and it seems very solid, so I'd consider this also. OpenHermes is amazing, so please put this variant of it in GPT4all soon. I could do the pull request if you want. I have a bit of time now.

[KeithCu]

I just noticed that merge above tops the openLLM leaderboard for the OpenHermes fine-tunes, so I think adding this model will be amazing and cause many people to rethink what smaller LLMs can do. It's far better than Mistral instruct, and far more fun and interesting and creative, and pretty solid at Python coding and problem solving.