llm icon indicating copy to clipboard operation
llm copied to clipboard
verified publisher icon rustformers

Implement MPT Model

Open danforbes opened this issue 2 years ago • 2 comments

Closes #214.

Tested with https://huggingface.co/LLukas22/mpt-7b-ggml

danforbes avatar May 12 '23 19:05 danforbes

I can't find a Replit model to test with.

danforbes avatar May 12 '23 19:05 danforbes

This is good to go, but will produce garbage output after a few tokens without this fix to ggml https://github.com/ggerganov/ggml/pull/145/files#diff-b4a500ab2765c31526c5541f3e51e21e46990b87d9774cac6f3089db315bdc5b

We'll have to wait for https://github.com/ggerganov/ggml/pull/145 to be merged.

philpax avatar May 14 '23 03:05 philpax

65k token context here we come

hhamud avatar May 18 '23 00:05 hhamud

Sounds like this is all good to go! @danforbes Does this include all of the changes from the ggml PR?

philpax avatar May 18 '23 13:05 philpax