flatsiedatsie
flatsiedatsie
This is becoming a bit of a show stopper unfortunately. It seems to even affect small models that would load under `llama_cpp_wasm` , such as [NeuralReyna](https://huggingface.co/afrideva/NeuralReyna-Mini-1.8B-v0.3-GGUF/resolve/main/neuralreyna-mini-1.8b-v0.3.q5_k_m.gguf) :-( If you could...
That's great news! Thank you so much!
The readme mentions the progress feature (very nice bonus, thank you!), but just to be sure: does this also address the memory issue? Or is the intended fix for that...
I'm seeing this error after creating a chunked model of Open Buddy Mistral 7B 32k Q4_K_M with 50 x 100Mb chunks: ``` "download_url":[ "https://huggingface.co/BoscoTheDog/open_buddy_mistral_7B_32k_chunked/resolve/main/open_buddy_mistral-00001-of-00050.gguf", "https://huggingface.co/BoscoTheDog/open_buddy_mistral_7B_32k_chunked/resolve/main/open_buddy_mistral-00002-of-00050.gguf", "https://huggingface.co/BoscoTheDog/open_buddy_mistral_7B_32k_chunked/resolve/main/open_buddy_mistral-00003-of-00050.gguf", "https://huggingface.co/BoscoTheDog/open_buddy_mistral_7B_32k_chunked/resolve/main/open_buddy_mistral-00004-of-00050.gguf", "https://huggingface.co/BoscoTheDog/open_buddy_mistral_7B_32k_chunked/resolve/main/open_buddy_mistral-00005-of-00050.gguf", "https://huggingface.co/BoscoTheDog/open_buddy_mistral_7B_32k_chunked/resolve/main/open_buddy_mistral-00006-of-00050.gguf", "https://huggingface.co/BoscoTheDog/open_buddy_mistral_7B_32k_chunked/resolve/main/open_buddy_mistral-00007-of-00050.gguf",...
I will. I've been trying lots of things actually. But unfortunately still having trouble loading models that WebLLM does load. The following screenshots are not so much bugs, I've managed...
I'm also still looking into your suggestion that it may be that the model is trying to load twice.
Correct, those are screenshots from yesterdy. I'm updating it now.
OK, I've done some more testing. TL/DR: Thing are running a lot smoother now! It's just the big models or big contexts that run out of memory. But before I...
Just a quick check: - Is it reasonable to set the `n_ctx` and `n_seq_max` to the same value? In the advanced example you only seem to set `n_ctx`. Do you...
*stops watching this space ;-)