LongLM icon indicating copy to clipboard operation
LongLM copied to clipboard

Published 20 hours ago • verified publisher icon datamllab

Question | Has anyone tried this with GGUF models?

Open Cognitus-Stuti opened this issue 10 months ago • 2 comments

Cognitus-Stuti avatar Apr 11 '24 23:04 Cognitus-Stuti

Llama.cpp has supported SelfExtend and had a good implementation. It uses GGUF models. SelfExtend has obtained pretty positive feedback from Llama.cpp's community. You can check their repo for more details.

Mooler0410 avatar Apr 12 '24 05:04 Mooler0410

probably, but i'd recommend to just make a .safetensors file Self-Extended before making it GGUF again

h-lunah avatar Aug 11 '24 07:08 h-lunah