unsloth icon indicating copy to clipboard operation
unsloth copied to clipboard

Published 20 hours ago • verified publisher icon unslothai

Support for Octpus LLM

Open avcode-exe opened this issue 1 year ago • 1 comments

Hi guys!, It will be nice to add support to Octopus LLMs or are they any alternative? The MMLU score of Octopus v4 is 74.8% under 5-shot, very impressive for such a small model! Octopus is based on Phi3 architecture, but it receive an error. 图片 图片

avcode-exe avatar May 19 '24 08:05 avcode-exe

Automatic model support is in the works!!

danielhanchen avatar May 19 '24 08:05 danielhanchen

hi @danielhanchen can I take this up ?

gjyotin305 avatar Jun 08 '24 12:06 gjyotin305

Wait this is a Phi-3 model but 128K - this requires first mistral-fying it, then adding the special RoPE methodology. @gjyotin305 Yes you're more than welcome to thanks!

danielhanchen avatar Jun 09 '24 14:06 danielhanchen

Ohk will look into it @danielhanchen

gjyotin305 avatar Jun 09 '24 16:06 gjyotin305