Mostelk
Mostelk
@farook-edev is it possible to truncate prompt 31 to 2048 (from the start, remove one or two input examples)
Loadgen min samples is 64 -> App will provide red color if you run less than 64, even with early stopping (5 min limit) Choices for max limit, for TinyMMLU...
Agreement to release on Github and PlayStore simultaneously, we may need to do Extended Testing on CI / browserstack before release/
Play Store/App Store per release (backend updates, major bug fixes, etc.): SOC Integration or Regular submission
Looks the release here https://github.com/mlcommons/mobile_app_open/releases/tag/v5.0 is older than playstore version @anhappdev ?
@farook-edev for 3.2 3B Llama model and 3.1 8B model, need to know if we can bring them in our implementation (dynamic quantized TFLite) to decide on the benchmark model
@anhappdev to provide updated list of devices on browserStack, to choose latest 2 most relevant ones per SOC vendor
https://www.browserstack.com/list-of-browsers-and-platforms/app_automate please check Galaxy Tab S10+ for 9300
Given we have this comprehensive test on BrowserStack, we can test this test again before every release, rather than every pull request.
@anhappdev he Browser Stack test is already run automatically when merged into the master branch. > Since this PR is already merged, https://github.com/mlcommons/mobile_app_open/pull/1049, did the full BrowserSTack CI/CD (expanded one)...