Skip to content

OpenVINO Model Server 2021.2
Compare
Choose a tag to compare
@dtrawins dtrawins released this 15 Dec 16:08
· 2072 commits to main since this release
v2021.2
5ab937c
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.

This is the second release of OVMS in C++ implementation. It includes several new features, enhancements and bug fixes. It contains as a backend OpenVINO Inference Engine in the same version - 2021.2.

New capabilities and enhancements

  • Directed Acyclic Graph (DAG) scheduler – (formerly models ensemble) this feature was first available as a preview in 2021.1. It is now officially supported, making it possible to define inference pipelines composed of multiple interconnected models that respond to a single prediction request. In this release we are adding support for remaining API calls which were not supported for DAGs in the preview, specifically GetModelStatus and GetModelMetadata. GetModelStatus returns the status of the complete pipeline while GetModelMetadata returns the pipeline inputs and outputs parameters. The new 2021.2 release has improved DAG config validation.
  • Direct import of ONNX models – it is now possible to serve ONNX models without converting to Intermediate Representation (IR) format. This helps simplify deployments using ONNX models and the PyTorch training framework.
  • Custom loaders and integration with OpenVINO™ Security Add-on – it is now possible to define a custom library to handle model loading operations – including additional steps related to model decryption and license verification. Review the documentation of the Security Add-on component to learn about controlled access to the models.
  • Traffic Encryption – new deployment recipe for client authorization via mTLS certificates and traffic encryption by integrating with NGINX reverse proxy in a Docker container.
  • Remote Model Caching from cloud storage – models stored in Google Cloud Storage (GCS), Amazon S3 and Azure blob will no longer be downloaded multiple times after configuration changes that require model reloading. Cached model(s) will be used during the model reload operation. When a served model is changed, only the corresponding new version folder will be added to the model storage.
  • updated versions of several third-party dependencies

Fixed bugs

  • Sporadic short unavailability of the default version when the model is switching to newer one
  • REST API not working with rest_workers=1 - there will be clear error message about invalid value. By default the number of REST worker threads will be adjusted automatically based on the CPUs
  • Prevented service crash when shape parameter is out of integer range

Known issues

  • version upgrade might fail when new model files are corrupted but older versions might be unloaded according to model version policy
  • ovms might sporadically fail under very heavy load on DAG execution during online update of pipeline models configuration. Predictions for individual models are not impacted.

You can use an OpenVINO Model Server public Docker image based on centos* via the following command:
docker pull openvino/model_server:2021.2 or
docker pull openvino/model_server:2021.2-gpu

Assets 2
Loading