Deep Neural Networks Benchmark for Android

Available on Google Play. Subscribe to news telegram channel. Discuss this repo, frameworks, networks on mobile in telegram group.

Road map

Frameworks:

• [x] MNN Alibaba ver. 1.0.2
• [x] NCNN Tencent ver. 20200727
• [x] TFLite Google ver. 2.3.0
• [x] TFMobile Google ver. 1.13.1
• [x] Pytorch Facebook ver. 1.5.0
• [x] OpenCV DNN ver. 4.4.0
• [x] onnxruntime Microsoft ver. 1.4.0 how to build
• [?] Mace by Xiaomi
• [ ] Tengine Lite OPEN AI LAB
• [ ] TNN Tencent
• [ ] NeoML ABBYY
• [?] SNPE Qualcomm
• [ ] HiAI Huawei
• [ ] NeuroPilot SDK Mediatek
• [ ] Paddle-Lite Baidu
• [ ] Samsung Neural SDK (if they approve my request)

Questionable/other:

• huawei-noah/bolt not very popular?
• JDAI-CV/dabnn binary networks

Features:

• [x] Compare inference results between frameworks and desktop
• [x] Visualize progress/results
• [x] Publish to Play Market
• [x] Collect results on backend
• [ ] Web site with agregated results

Models:

• [x] MobileNet v2
• [x] Deeplab v3
• [ ] Bert Question and Answer

All models are floating point

Supported ABIs: armeabi-v7a, arm64-v8a. Some frameworks (eg TF) also supports x86 and x86_64, but are they still exist in 2020?

Conversions / Run your own model

A detailed explanation of how to convert the model into each framework available here.

Here is repo with docker images contatining some built converters and other maybe nessesary tools. gordinmitya/docker_that_framework

Credits

• Thanks to Rohithkvsp/OnnxRuntimeAndorid for sample code on how to use onnxruntime with nnapi!

SNPE

Qualcomm prohibits redestribution of their libraries, so you have to register there and download them by yourself. ¯\_(ツ)_/¯

1. Register and download zip from developer.qualcomm.com;
2. Copy android/snpe-release.aar from archive into snpe/libs.

OR compile without snpe

1. Remove , ':snpe' from settings.gradle;
2. Remove implementation project(path: ':snpe') from app/build.gradle;
3. Remove amy mentions of SNPE in MainActivity.kt.

License Summary

Project itself and code inside ru.gordinmitya.* packages are under MIT licanse as stated in LICENSE file.

Code inside other packages (eg org.opencv.*) or some C plus plus code may be under other licenses.

RANDOM

ImageNet samples were taken from Kaggle.