merge master@2f47262 documents to dev-1.x

open-mmlab · Sep 15, 2022 · 3a925b6 · 3a925b6
1 parent 06028d6
commit 3a925b6
Show file tree

Hide file tree

Showing 112 changed files with 6,588 additions and 1,003 deletions.
diff --git a/.codespell_ignore.txt b/.codespell_ignore.txt
@@ -0,0 +1,2 @@
+cann
+CANN
diff --git a/README.md b/README.md
@@ -55,33 +55,45 @@ The currently supported codebases and models are as follows, and more will be in
 
 Models can be exported and run in the following backends, and more will be compatible
 
-| ONNX Runtime | TensorRT | ppl.nn | ncnn | OpenVINO | LibTorch | snpe | more                                           |
-| ------------ | -------- | ------ | ---- | -------- | -------- | ---- | ---------------------------------------------- |
-| ✔️           | ✔️       | ✔️     | ✔️   | ✔️       | ✔️       | ✔️   | [benchmark](docs/en/03-benchmark/benchmark.md) |
+| ONNX Runtime | TensorRT | ppl.nn | ncnn | OpenVINO | LibTorch | snpe | Ascend | Core ML | RKNN | more                                           |
+| ------------ | -------- | ------ | ---- | -------- | -------- | ---- | ------ | ------- | ---- | ---------------------------------------------- |
+| ✔️           | ✔️       | ✔️     | ✔️   | ✔️       | ✔️       | ✔️   | ✔️     | ✔️      | ✔️   | [benchmark](docs/en/03-benchmark/benchmark.md) |
 
 ### Efficient and scalable C/C++ SDK Framework
 
 All kinds of modules in the SDK can be extended, such as `Transform` for image processing, `Net` for Neural Network inference, `Module` for postprocessing and so on
 
-## Get Started
+## [Documentation](https://mmdeploy.readthedocs.io/en/latest/)
 
-Please read [getting_started.md](docs/en/get_started.md) for the basic usage of MMDeploy. We also provide tutoials about:
+Please read [getting_started](docs/en/get_started.md) for the basic usage of MMDeploy. We also provide tutoials about:
 
 - [Build](docs/en/01-how-to-build/build_from_source.md)
   - [Build from Docker](docs/en/01-how-to-build/build_from_docker.md)
+  - [Build from Script](docs/en/01-how-to-build/build_from_script.md)
   - [Build for Linux](docs/en/01-how-to-build/linux-x86_64.md)
+  - [Build for macOS](docs/en/01-how-to-build/macos-arm64.md)
   - [Build for Win10](docs/en/01-how-to-build/windows.md)
   - [Build for Android](docs/en/01-how-to-build/android.md)
   - [Build for Jetson](docs/en/01-how-to-build/jetsons.md)
   - [Build for SNPE](docs/en/01-how-to-build/snpe.md)
 - User Guide
   - [How to convert model](docs/en/02-how-to-run/convert_model.md)
   - [How to write config](docs/en/02-how-to-run/write_config.md)
-  - [How to evaluate deployed models](docs/en/02-how-to-run/how_to_evaluate_a_model.md)
-  - [How to measure performance of deployed models](docs/en/02-how-to-run/how_to_measure_performance_of_models.md)
+  - [How to profile model](docs/en/02-how-to-run/profile_model.md)
+  - [How to quantize model](docs/en/02-how-to-run/quantize_model.md)
+  - [Useful tools](docs/en/02-how-to-run/useful_tools.md)
 - Developer Guide
-  - [How to support new models](docs/en/06-developer-guide/support_new_model.md)
-  - [How to support new backends](docs/en/06-developer-guide/support_new_backend.md)
+  - [Architecture](docs/en/07-developer-guide/architecture.md)
+  - [How to support new models](docs/en/07-developer-guide/support_new_model.md)
+  - [How to support new backends](docs/en/07-developer-guide/support_new_backend.md)
+  - [How to partition model](docs/en/07-developer-guide/partition_model.md)
+  - [How to test rewritten model](docs/en/07-developer-guide/test_rewritten_models.md)
+  - [How to test backend ops](docs/en/07-developer-guide/add_backend_ops_unittest.md)
+  - [How to do regression test](docs/en/07-developer-guide/regression_test.md)
+- Custom Backend Ops
+  - [ncnn](docs/en/06-custom-ops/ncnn.md)
+  - [onnxruntime](docs/en/06-custom-ops/onnxruntime.md)
+  - [tensorrt](docs/en/06-custom-ops/tensorrt.md)
 - [FAQ](docs/en/faq.md)
 - [Contributing](.github/CONTRIBUTING.md)
 

diff --git a/README_zh-CN.md b/README_zh-CN.md
@@ -53,21 +53,24 @@ MMDeploy 是 [OpenMMLab](https://openmmlab.com/) 模型部署工具箱，**为
 
 ### 支持多种推理后端
 
-| ONNX Runtime | TensorRT | ppl.nn | ncnn | OpenVINO | LibTorch | snpe | more                                              |
-| ------------ | -------- | ------ | ---- | -------- | -------- | ---- | ------------------------------------------------- |
-| ✔️           | ✔️       | ✔️     | ✔️   | ✔️       | ✔️       | ✔️   | [benchmark](docs/zh_cn/03-benchmark/benchmark.md) |
+| ONNX Runtime | TensorRT | ppl.nn | ncnn | OpenVINO | LibTorch | snpe | Ascend | Core ML | RKNN | more                                           |
+| ------------ | -------- | ------ | ---- | -------- | -------- | ---- | ------ | ------- | ---- | ---------------------------------------------- |
+| ✔️           | ✔️       | ✔️     | ✔️   | ✔️       | ✔️       | ✔️   | ✔️     | ✔️      | ✔️   | [benchmark](docs/en/03-benchmark/benchmark.md) |
 
 ### SDK 可高度定制化
 
 - Transform 数据预处理
 - Net 推理
 - Module 后处理
 
-## [快速上手](docs/zh_cn/get_started.md)
+## [中文文档](https://mmdeploy.readthedocs.io/zh_CN/latest/)
 
+- [快速上手](docs/zh_cn/get_started.md)
 - [编译](docs/zh_cn/01-how-to-build/build_from_source.md)
+  - [一键式脚本安装](docs/zh_cn/01-how-to-build/build_from_script.md)
   - [Build from Docker](docs/zh_cn/01-how-to-build/build_from_docker.md)
   - [Build for Linux](docs/zh_cn/01-how-to-build/linux-x86_64.md)
+  - [Build for macOS](docs/zh_cn/01-how-to-build/macos-arm64.md)
   - [Build for Win10](docs/zh_cn/01-how-to-build/windows.md)
   - [Build for Android](docs/zh_cn/01-how-to-build/android.md)
   - [Build for Jetson](docs/en/01-how-to-build/jetsons.md)
@@ -77,17 +80,29 @@ MMDeploy 是 [OpenMMLab](https://openmmlab.com/) 模型部署工具箱，**为
   - [配置转换参数](docs/zh_cn/02-how-to-run/write_config.md)
   - [量化](docs/zh_cn/02-how-to-run/quantize_model.md)
   - [测试转换完成的模型](docs/zh_cn/02-how-to-run/profile_model.md)
+  - [工具集介绍](docs/zh_cn/02-how-to-run/useful_tools.md)
 - 开发指南
-  - [支持新模型](docs/zh_cn/04-developer-guide/support_new_model.md)
-  - [增加推理 Backend](docs/zh_cn/04-developer-guide/support_new_backend.md)
-  - [回归测试](docs/zh_cn/04-developer-guide/do_regression_test.md)
+  - [软件架构](docs/zh_cn/07-developer-guide/architecture.md)
+  - [支持新模型](docs/zh_cn/07-developer-guide/support_new_model.md)
+  - [增加推理 backend](docs/zh_cn/07-developer-guide/support_new_backend.md)
+  - [模型分块](docs/zh_cn/07-developer-guide/partition_model.md)
+  - [测试重写模型](docs/zh_cn/07-developer-guide/test_rewritten_models.md)
+  - [backend 算子测试](docs/zh_cn/07-developer-guide/add_backend_ops_unittest.md)
+  - [回归测试](docs/zh_cn/07-developer-guide/regression_test.md)
+- 各 backend 自定义算子列表
+  - [ncnn](docs/zh_cn/06-custom-ops/ncnn.md)
+  - [onnxruntime](docs/zh_cn/06-custom-ops/onnxruntime.md)
+  - [tensorrt](docs/zh_cn/06-custom-ops/tensorrt.md)
 - [FAQ](docs/zh_cn/faq.md)
 - [贡献者手册](.github/CONTRIBUTING.md)
 
 ## 新人解说
 
-- [01 术语解释、加载第一个模型](docs/zh_cn/05-tutorial/01_introduction_to_model_deployment.md)
-- [02 转成 onnx](docs/zh_cn/05-tutorial/02_challenges.md)
+- [01 术语解释、加载第一个模型](docs/zh_cn/tutorial/01_introduction_to_model_deployment.md)
+- [02 部署常见问题](docs/zh_cn/tutorial/02_challenges.md)
+- [03 torch转onnx](docs/zh_cn/tutorial/03_pytorch2onnx.md)
+- [04 让torch支持更多onnx算子](docs/zh_cn/tutorial/04_onnx_custom_op.md)
+- [05 调试onnx模型](docs/zh_cn/tutorial/05_onnx_model_editing.md)
 
 ## 基准与模型库
 

diff --git a/docs/en/01-how-to-build/android.md b/docs/en/01-how-to-build/android.md
@@ -17,7 +17,7 @@ Model converter is executed on linux platform, and SDK is executed on android pl
 
 Here are two steps for android build.
 
-1. Build model converter on linux, please refer to [How to build linux](./linux-x86_64.md)
+1. Build model converter on linux, please refer to [How to build linux](linux-x86_64.md)
 
 2. Build SDK using android toolchain on linux.
 
@@ -106,73 +106,6 @@ make -j$(nproc) install
 
 ### Build MMDeploy
 
-#### Build Options Spec
-
-<table>
-<thead>
-  <tr>
-    <th>NAME</th>
-    <th>VALUE</th>
-    <th>DEFAULT</th>
-    <th>REMARK</th>
-  </tr>
-</thead>
-<tbody>
-  <tr>
-    <td>MMDEPLOY_BUILD_SDK</td>
-    <td>{ON, OFF}</td>
-    <td>OFF</td>
-    <td>Switch to build MMDeploy SDK</td>
-  </tr>
-  <tr>
-    <td>MMDEPLOY_BUILD_SDK_PYTHON_API</td>
-    <td>{ON, OFF}</td>
-    <td>OFF</td>
-    <td>switch to build MMDeploy SDK python package</td>
-  </tr>
-  <tr>
-    <td>MMDEPLOY_BUILD_SDK_JAVA_API</td>
-    <td>{ON, OFF}</td>
-    <td>OFF</td>
-    <td>switch to build MMDeploy SDK Java API</td>
-  </tr>
-  <tr>
-    <td>MMDEPLOY_BUILD_TEST</td>
-    <td>{ON, OFF}</td>
-    <td>OFF</td>
-    <td>Switch to build MMDeploy SDK unittest cases</td>
-  </tr>
-  <tr>
-    <td>MMDEPLOY_TARGET_DEVICES</td>
-    <td>{"cpu"}</td>
-    <td>cpu</td>
-    <td>Enable target device. <br>If you want use ncnn vulkan accelerate, you still fill <code>{"cpu"}</code> here. Because, vulkan accelerate is only for ncnn net. The other part of inference is still using cpu.</td>
-  </tr>
-  <tr>
-    <td>MMDEPLOY_TARGET_BACKENDS</td>
-    <td>{"ncnn"}</td>
-    <td>N/A</td>
-    <td>Enabling inference engine. <br><b>By default, no target inference engine is set, since it highly depends on the use case.</b><br> Only ncnn backend is supported for android platform now.<br>
-    After specifying the inference engine, it's package path has to be passed to cmake as follows, <br>
-    1. <b>ncnn</b>: ncnn. <code>ncnn_DIR</code> is needed.
-<pre><code>-Dncnn_DIR=${NCNN_DIR}/build_${ANDROID_ABI}/install/lib/cmake/ncnn</code></pre>
-   </td>
-  </tr>
-  <tr>
-    <td>MMDEPLOY_CODEBASES</td>
-    <td>{"mmcls", "mmdet", "mmseg", "mmedit", "mmocr", "all"}</td>
-    <td>N/A</td>
-    <td>Enable codebase's postprocess modules. It MUST be set by a semicolon separated list of codebase names. The currently supported codebases are 'mmcls', 'mmdet', 'mmedit', 'mmseg', 'mmocr'. Instead of listing them one by one, you can also pass <code>all</code> to enable them all, i.e., <code>-DMMDEPLOY_CODEBASES=all</code>Please manually edit <code>csrc/mmdeploy/apis/java/native/CMakeLists.txt</code> to avoid compilation errors.</td>
-  </tr>
-  <tr>
-    <td>MMDEPLOY_SHARED_LIBS</td>
-    <td>{ON, OFF}</td>
-    <td>ON</td>
-    <td>Switch to build shared library or static library of MMDeploy SDK. Now you should build static library for android. Bug will be fixed soon.</td>
-  </tr>
-</tbody>
-</table>
-
 #### Build SDK and Demos
 
 MMDeploy provides a recipe as shown below for building SDK with ncnn as inference engine for android.
@@ -189,7 +122,6 @@ MMDeploy provides a recipe as shown below for building SDK with ncnn as inferenc
       -DOpenCV_DIR=${OPENCV_ANDROID_SDK_DIR}/sdk/native/jni/abi-${ANDROID_ABI} \
       -Dncnn_DIR=${NCNN_DIR}/build_${ANDROID_ABI}/install/lib/cmake/ncnn \
       -DMMDEPLOY_TARGET_BACKENDS=ncnn \
-      -DMMDEPLOY_CODEBASES=all \
       -DMMDEPLOY_SHARED_LIBS=ON \
       -DCMAKE_TOOLCHAIN_FILE=${NDK_PATH}/build/cmake/android.toolchain.cmake \
       -DANDROID_ABI=${ANDROID_ABI} \
@@ -198,3 +130,5 @@ MMDeploy provides a recipe as shown below for building SDK with ncnn as inferenc
 
   make -j$(nproc) && make install
   ```
+
+Please check [cmake build option spec](cmake_option.md)
diff --git a/docs/en/01-how-to-build/build_from_docker.md b/docs/en/01-how-to-build/build_from_docker.md
@@ -51,7 +51,7 @@ docker run --gpus all -it mmdeploy:master-gpu
 
    As described [here](https://forums.developer.nvidia.com/t/cuda-error-the-provided-ptx-was-compiled-with-an-unsupported-toolchain/185754), update the GPU driver to the latest one for your GPU.
 
-2. docker: Error response from daemon: could not select device driver "" with capabilities: \[\[gpu\]\].
+2. docker: Error response from daemon: could not select device driver "" with capabilities: \[gpu\].
 
    ```
    # Add the package repositories

diff --git a/docs/en/01-how-to-build/build_from_script.md b/docs/en/01-how-to-build/build_from_script.md
@@ -0,0 +1,45 @@
+# Build from Script
+
+Through user investigation, we know that most users are already familiar with python and torch before using mmdeploy. Therefore we provide scripts to simplify mmdeploy installation.
+
+Assuming you have a python ready (whether `conda` or `pyenv`), run this script to install mmdeploy + ncnn backend, `nproc` is not compulsory.
+
+```bash
+$ cd /path/to/mmdeploy
+$ python3 tools/scripts/build_ubuntu_x64_ncnn.py $(nproc)
+..
+```
+
+A sudo password may be required during this time, and the script will try its best to build and install mmdeploy SDK and demo:
+
+- Detect host OS version, `make` job number, whether use `root` and try to fix `python3 -m pip`
+- Find the necessary basic tools, such as g++-7, cmake, wget, etc.
+- Compile necessary dependencies, such as pyncnn, protobuf
+
+The script will also try to avoid affecting host environment:
+
+- The dependencies of source code compilation are placed in the `mmdeploy-dep` directory at the same level as mmdeploy
+- The script would not modify variables such as PATH, LD_LIBRARY_PATH, PYTHONPATH, etc.
+- The environment variables that need to be modified will be printed, **please pay attention to the final output**
+
+The script will eventually execute `python3 tools/check_env.py`, the successful installation should display the version number of the corresponding backend and `ops_is_available: True`, for example:
+
+```bash
+$ python3 tools/check_env.py
+..
+2022-09-13 14:49:13,767 - mmdeploy - INFO - **********Backend information**********
+2022-09-13 14:49:14,116 - mmdeploy - INFO - onnxruntime: 1.8.0	ops_is_avaliable : True
+2022-09-13 14:49:14,131 - mmdeploy - INFO - tensorrt: 8.4.1.5	ops_is_avaliable : True
+2022-09-13 14:49:14,139 - mmdeploy - INFO - ncnn: 1.0.20220901	ops_is_avaliable : True
+2022-09-13 14:49:14,150 - mmdeploy - INFO - pplnn_is_avaliable: True
+..
+```
+
+Here is the verified installation script. If you want mmdeploy to support multiple backends at the same time, you can execute each script once:
+
+|             script              | OS version  |
+| :-----------------------------: | :---------: |
+|    build_ubuntu_x64_ncnn.py     | 18.04/20.04 |
+|     build_ubuntu_x64_ort.py     | 18.04/20.04 |
+|    build_ubuntu_x64_pplnn.py    | 18.04/20.04 |
+| build_ubuntu_x64_torchscript.py | 18.04/20.04 |
diff --git a/docs/en/01-how-to-build/build_from_source.md b/docs/en/01-how-to-build/build_from_source.md
@@ -1,4 +1,4 @@
-# Build From Source
+# Build from Source
 
 ## Download
 
@@ -38,3 +38,4 @@ Please visit the following links to find out how to build MMDeploy according to
 - [Android-aarch64](android.md)
 - [NVIDIA Jetson](jetsons.md)
 - [SNPE](snpe.md)
+- [RISC-V](riscv.md)