Huggingface Automodel compliant LLAMA model

[PerpSeal]

Hey guys its a bit unclear what expected here : m = GPT4AllGPU(LLAMA_PATH)

Where do I get that? Which kind of file is it ?

[Aunxfb]

Something from here: https://huggingface.co/models

[Slowly-Grokking]

I don't have this working yet either but to try to be more helpful, i think you specifically need something from here: https://huggingface.co/models?other=llama

edit: I made a little progress:

git lfs install git clone <huggingface.co link> (such as .../zpn/llama-7b)

check the config.json and tokenizer_config.json for improper capitalization. (https://github.com/huggingface/transformers/issues/22222#issuecomment-1477171703) "Change the LLaMATokenizer in tokenizer_config.json into lowercase LlamaTokenizer and it works like a charm."

[keodarus]

The thread: https://github.com/nomic-ai/gpt4all/issues/159#issue-1650481558 suggests using the model https://huggingface.co/decapoda-research/llama-7b-hf

Thanks to @Slowly-Grokking, it worked for me. After setting the path in the nomic/gpt4allGPU.py file to where the cloned huggingface repository resides e.g. m = GPT4AllGPU('/home/user/gpt4all/llama-7b-hf') (make sure to enclose the path within quotation marks) and changing "tokenizer_class": "LLaMATokenizer" to "tokenizer_class": "LlamaTokenizer" in the llama-7b-hf/tokenizer_config.json file, it starts to work in a technical sense.

Note that using an LLaMA model from Huggingface (which is Hugging Face Automodel compliant and therefore GPU acceleratable by gpt4all) means that you are no longer using the original assistant-style fine-tuned, quantized LLM LoRa.

[niansa]

Stale, please open a new issue if this is still relevant.