WizardCoder llama assert failure

Open jacohend opened this issue 2 years ago • 3 comments

Trying to run a variety of ggml models from TheBloke leads to this error: GGML_ASSERT: llama-cpp/ggml.c:6270: ggml_nelements(a) == ne0*ne1*ne2

Wondering if anyone else is experiencing this, and what the issue might be?

Aug 28 '23 04:08 jacohend

Related: https://github.com/ggerganov/llama.cpp/issues/2445#issuecomment-1656949448

Aug 28 '23 05:08 jacohend

Probably another issue with the currently used ggml version, a re-sync with the current main branch of llama.cpp is probably needed.

Aug 28 '23 08:08 LLukas22

I actually did that and found a failure on the same assert line. The linked comment said rolling the version back worked best.

I'm wondering if this assert is assuming constant layer sizes, so any modification like The Bloke does might be causing the failure?

Aug 28 '23 14:08 jacohend