ggml v3 support

[yesbroc]

Feature request

support for ggml v3 for q4 and q8 models (also some q5 from thebloke)

Motivation

the best models are being quantized in v3 e.g airoboros, manticore, and guanaco

Your contribution

there is no way i can help

[AndriyMulyar]

The next release will support this. Changes are now in the main branch :)

[niansa]

Done