Austin Huang

Hi @trholding thanks for the suggestions! I'm looking into options for accelerator support, ideally while still keeping things simple + not expanding the dependency footprint too much - thanks for...

Closing for now, but yes track https://github.com/google/gemma.cpp/issues/18 will add some additional info in the README as well.

it won't always be monotonically increasing with # threads, can be quite system dependent so takes a bit of experimentation. You might want to try 2b-it-sfp which should be faster...

Closing for now, if there's anything that's not addressed above, feel free to chime in. Also added a small note to the README "What are some easy ways to make...

Hi @justinsb thanks for taking the initiative. There's been interest in client/server capabilities and I think there's some obvious use cases + value in that. There's a few things being...

@esrrhs thanks for reporting. Will close for now but @bigbigzxl if upgrading binutils doesn't resolve this for you, can post here + reopen.

@purefire adding an export script is a high priority to unlock model variations / fine tuning. Please chime in https://github.com/google/gemma.cpp/issues/11 on the source format that would be most useful. Hi...

Consolidating weight export discussions to the issue https://github.com/google/gemma.cpp/issues/11 . We're working on making a script available soon - feel free to chime in with further comments there. Thanks!

This is a great question. There's two big TODOs that will make using gemma.cpp as a library much better: - We have some example demo applications in the works. They're...

I'm working on adding an example of using libgemma here: https://github.com/google/gemma.cpp/pull/82 in addition to refactoring library usage. Still a bit more work before merging, but if you're interested in libgemma,...