schmorp
schmorp
For the 340B, conversion instantly fails flat because there isn't a config.json file.
And just to document this here, Llama-3.1-Minitron-4B-Width-Base fails with: cvs/llama.cpp/ggml/src/ggml.c:6399: GGML_ASSERT(c->ne[0] >= n_dims / 2) failed
This still seems to be the case in 24.10.03. The device starts blinking rapidly, it responds to pings to 192.168.1.1 three times (not the normally configured address), but neither port...
It might or might not be this issue, but a way to reproduce this is that, if a torrent file is complete, but transmission fails to rename it for some...
Why couldn't this tensor be added by llama.cpp when loading? superficially it doesn't make much sense to bake the rope config into the model at conversion time, and prevents bugfixes...