Koan-Sin Tan issues

Repositories
Issues
Comments

Results 32 issues of


                                            Koan-Sin Tan

"Accuracy" metric for LLM model(s)

104

Which "accuracy" metric(s) should we use for LLM benchmarking? - MMLU: the first item people choose. it covers several field with multiple-choice questions. - @mohitmundhragithub please point where/how MLPerf Client...

Check if we can leverage Google Play's "Play for On-device AI"

Google Play's [Play for On-device AI"](https://developer.android.com/google/play/on-device-ai) looks convenient, However, - can we use it to provide on-demand delivery of non-tflite models (seems to be yes) - how about iOS app?