Deep Neural Networks Benchmark for Android

Available on Google Play. Subscribe to news telegram channel. Discuss this repo, frameworks, networks on mobile in telegram group.

Road map

Frameworks:

• MNN Alibaba ver. 1.0.2
• NCNN Tencent ver. 20200727
• TFLite Google ver. 2.3.0
• TFMobile Google ver. 1.13.1
• Pytorch Facebook ver. 1.5.0
• OpenCV DNN ver. 4.4.0
• onnxruntime Microsoft ver. 1.4.0 how to build
• [?] Mace by Xiaomi
• Tengine Lite OPEN AI LAB
• TNN Tencent
• NeoML ABBYY
• [?] SNPE Qualcomm
• HiAI Huawei
• NeuroPilot SDK Mediatek
• Paddle-Lite Baidu
• Samsung Neural SDK (if they approve my request)

Questionable/other:

• huawei-noah/bolt not very popular?
• JDAI-CV/dabnn binary networks

Features:

• Compare inference results between frameworks and desktop
• Visualize progress/results
• Publish to Play Market
• Collect results on backend
• Web site with agregated results

Models:

• MobileNet v2
• Deeplab v3
• Bert Question and Answer

All models are floating point

Supported ABIs: armeabi-v7a, arm64-v8a. Some frameworks (eg TF) also supports x86 and x86_64, but are they still exist in 2020?

Conversions / Run your own model

A detailed explanation of how to convert the model into each framework available here.

Here is repo with docker images contatining some built converters and other maybe nessesary tools. gordinmitya/docker_that_framework

Credits

• Thanks to Rohithkvsp/OnnxRuntimeAndorid for sample code on how to use onnxruntime with nnapi!

SNPE

Qualcomm prohibits redestribution of their libraries, so you have to register there and download them by yourself. ¯\_(ツ)_/¯

1. Register and download zip from developer.qualcomm.com;
2. Copy android/snpe-release.aar from archive into snpe/libs.

OR compile without snpe

1. Remove , ':snpe' from settings.gradle;
2. Remove implementation project(path: ':snpe') from app/build.gradle;
3. Remove amy mentions of SNPE in MainActivity.kt.

License Summary

Project itself and code inside ru.gordinmitya.* packages are under MIT licanse as stated in LICENSE file.

Code inside other packages (eg org.opencv.*) or some C plus plus code may be under other licenses.

RANDOM

ImageNet samples were taken from Kaggle.