Austin Huang
Austin Huang
Closing for now but will update here when there's more info on this.
Great suggestion, if there's others who interested please +emoji above and we'll prioritize this :)
Hi @sanjay920, really cool that you're trying a fine tune already. We're working on releasing a conversion script soon (hopefully within the next few days), but would be useful to...
Understood, the -sfp models are 8 bit weights, but I understand people are interested in more aggressive quantization. BTW for just decreasing the memory footprint there was a commit that...
ouch sorry about that, will take a closer look a bit later this evening.
Hi @Code-keys , can you try with 2b-it-sfp.sbs? SFP uses compressed 8bit weights. also make sure to use the right build (if you use 2b-it vs. 2b-it-sfp there's a different...
Re memory issues: I'm going to adjust some defaults + make kSeqLen configurable that should improve the situation a bit. In configs.h, a key parameter kSeqLen which preallocates a kv...
I think this is related to the namespace issue, which should be resolved in this commit on the dev branch: https://github.com/google/gemma.cpp/commit/4a0d23f47ee36370e8db429648f58af6fdb9f953 Might try with the dev branch or alternatively I'll...
I agree cmake can be painful but the landscape of build systems + package management for C++ ... leaves something to be desired and for all its faults cmake seems...
If @jart is fine with it, what if we adapt that Makefile + submodules for local builds? If anyone wants to submit a PR I'm happy to review. Has anyone...