Philpax
Philpax
Looks like someone's going for it https://github.com/saharNooby/rwkv.cpp
Interesting! Yes, we're open to having our own fork of GGML - we'll just have to manage the patches ourselves. The primary reason that I think they would shy away...
This will need to be built on top of #84.
There are some changes in #141 around loading infrastructure. Wait for that one to be merged first.
There are models floating around that aren't fully compliant with the format (e.g. missing scores). This tool could be used as a general-purpose solution for reading in those models with...
Closing this because multipart GGML models are pretty rare now.
@RedBoxing has taken the lead on this 👀 Do you want to open a PR and I'll close mine?
I'm looking into the quantisation issue. Would you be able to update the models to use `inplace`? https://github.com/ggerganov/ggml/commit/3ce314550126a611bd6294da193380d51b8df86d
Realized when I woke up this morning that the quantization-check will fail on v1 f16 models, which should work (as they don't have any quantization). Will need to fix after...
@danforbes are you OK with the state of the PR as-is? Might merge it and do the de/requant stuff in another PR to unblock this.