Mostelk
Mostelk
Shall we use this commit for the release? https://github.com/mlcommons/mobile_app_open/commit/0e726d86f125515ddfde62c3f72f6b71c96ac804
> > Shall we use this commit for the release? [0e726d8](https://github.com/mlcommons/mobile_app_open/commit/0e726d86f125515ddfde62c3f72f6b71c96ac804) > > Yes, it should be the latest version with the correct app version. In that case, please also...
> I agree with Anh that the releases notes can go into the Play Store and GitHub listings and don't need to be embedded in the app. > > For...
@mohitmundhragithub thanks for checking, this is also our observation, I dont know of any special setting, but SW Android updates may vary the result, due to thermal control which makes...
@anhappdev @freedomtan The PlayStore version is crashing on FLIP 7 but the released 5.0.2 version is working fine, looks there is some problem with PlayStore as we had before?
@anhappdev @mohitmundhragithub I just tested the Play Store App on S23 Ultra as well with Snapdragon backend, it also crashed once GO is pressed.
Let us try to quantize these and report accuracy llama 3.1 8B Instruct, llama 3.2 3B Instruct
@mohitmundhragithub @freedomtan I agree let us consider other metrics as well, but I prefer one metric that is publicly published by Meta, for example I see here they use SQUAD...
@mohitmundhragithub @freedomtan please check these 3.1 8B quantized models, with their evaluation, looks using lm-eval, so may be we adopt some tiny lm-eval ( appears using static weight quantization, but...