MMPose aka mmpose is an open-source toolbox for pose estimation based on PyTorch. It is a part of the OpenMMLab project.
Please follow the best practice to install mmpose.
There are several methods to install mmdeploy, among which you can choose an appropriate one according to your target platform and device.
Method I: Install precompiled package
TODO. MMDeploy hasn't released based on dev-1.x branch.
Method II: Build using scripts
If your target platform is Ubuntu 18.04 or later version, we encourage you to run
scripts. For example, the following commands install mmdeploy as well as inference engine - ONNX Runtime.
git clone --recursive -b dev-1.x https://github.com/open-mmlab/mmdeploy.git
cd mmdeploy
python3 tools/scripts/build_ubuntu_x64_ort.py $(nproc)
export PYTHONPATH=$(pwd)/build/lib:$PYTHONPATH
export LD_LIBRARY_PATH=$(pwd)/../mmdeploy-dep/onnxruntime-linux-x64-1.8.1/lib/:$LD_LIBRARY_PATHMethod III: Build from source
If neither I nor II meets your requirements, building mmdeploy from source is the last option.
You can use tools/deploy.py to convert mmpose models to the specified backend models. Its detailed usage can be learned from here.
The command below shows an example about converting hrnet model to onnx model that can be inferred by ONNX Runtime.
It is crucial to specify the correct deployment config during model conversion. We've already provided builtin deployment config files of all supported backends for mmpose. The config filename pattern is:
pose-detection_{backend}-{precision}_{static | dynamic}_{shape}.py
- {backend}: inference backend, such as onnxruntime, tensorrt, pplnn, ncnn, openvino, coreml etc.
- {precision}: fp16, int8. When it's empty, it means fp32
- {static | dynamic}: static shape or dynamic shape
- {shape}: input shape or shape range of a model
Therefore, in the above example, you can also convert hrnet to other backend models by changing the deployment config file pose-detection_onnxruntime_static.py to others, e.g., converting to tensorrt model by pose-detection_tensorrt_static-256x192.py.
When converting mmpose models to tensorrt models, --device should be set to "cuda"
Before moving on to model inference chapter, let's know more about the converted model structure which is very important for model inference.
The converted model locates in the working directory like mmdeploy_models/mmpose/ort in the previous example. It includes:
mmdeploy_models/mmpose/ort
├── deploy.json
├── detail.json
├── end2end.onnx
└── pipeline.json
in which,
- end2end.onnx: backend model which can be inferred by ONNX Runtime
- deploy.json: meta information about backend model
- pipeline.json: inference pipeline of mmdeploy SDK
- detail.json: conversion parameters
The whole package mmdeploy_models/mmpose/ort is defined as mmdeploy SDK model, i.e., mmdeploy SDK model includes both backend model and inference meta information.
MMDeploy provides a unified API named as inference_model to inference model, making all inference backends API transparent to users.
Take the previous converted end2end.onnx model as an example, you can use the following code to inference the model and visualize the results.
TODO
| Model | Task | ONNX Runtime | TensorRT | ncnn | PPLNN | OpenVINO |
|---|---|---|---|---|---|---|
| HRNet | PoseDetection | Y | Y | Y | N | Y |
| MSPN | PoseDetection | Y | Y | Y | N | Y |
| LiteHRNet | PoseDetection | Y | Y | Y | N | Y |