Koan-Sin Tan
Koan-Sin Tan
Let's branch out v4_0, and try to add the three things above.
Thanks @anhappdev for merging all the necessary patches into the submission-v4.0 branch. @mohitmundhragithub and @AhmedTElthakeb Please help check if it is OK https://github.com/mlcommons/mobile_app_open/tree/submission-v4.0
- [x] mediatek - [x] qualcomm - [x] samsung: it should be fine. need check again. @AhmedTElthakeb to confirm it.
@anhappdev to create an issue for loadgen
> @freedomtan please share the info how to check the model accuracy for the Mobilenet V4. What dataset do I need to use, and if we have some specific steps...
@RSMNYS I don't get it. This is the original Mobielnet EdgeTPU model we had or a new V4 one? As far as I can remember, we checked that we can...
FYR, on an iPhone 13, for Mobilenet EdgeTPU I got 76.21% running binary built from lastest master branch.
@RSMNYS thermal throttling is a well-known issue on cell phone. A typical way to get numbers we want is to cool down the device before you run a new test...
please try to do the first 3 items and ensure that there is not thermal throttling. e.g., cold start, wait for 5 mins, and measure the performance numbers. Note that...
> All tests were done on iPhone 14 Pro > > Model Name Performance (QPS) Accuracy (%) Size, Mb > MobilenetV4_Large.mlmodel 294.85 81.2% 124 > MobilenetV4_Large.mlpackage 296.93 82.73% 65.5 >...