Skip to content

Commit

Permalink
Render KServe Python Runtime API doc with mkdoc (#333)
Browse files Browse the repository at this point in the history 
* Update KServe python sdk docs

Signed-off-by: Dan Sun <[email protected]>

* Update serving runtime doc

Signed-off-by: Dan Sun <[email protected]>

---------

Signed-off-by: Dan Sun <[email protected]>
  • Loading branch information
@yuzisun
yuzisun authored Dec 31, 2023
1 parent 82bacd2 commit d20e3a3
Show file tree
Hide file tree
Showing 5 changed files with 104 additions and 67 deletions.
28 changes: 14 additions & 14 deletions docs/modelserving/v1beta1/serving_runtime.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,20 +33,20 @@ These can also be found in the respective [runtime YAML](https://github.com/kser
For model frameworks using the KServe serving runtime, the specific default version can be found in [kserve/python](https://github.com/kserve/kserve/tree/master/python).
In a given serving runtime directory the pyproject.toml file contains the exact model framework version used. For example, in [kserve/python/lgbserver](https://github.com/kserve/kserve/tree/master/python/lgbserver) the [pyproject.toml](https://github.com/kserve/kserve/blob/master/python/lgbserver/pyproject.toml) file sets the model framework version to 3.3.2, `lightgbm ~= 3.3.2`.

| Model Serving Runtime | Exported model | HTTP | gRPC | Default Serving Runtime Version | Supported Framework (Major) Version(s) | Examples |
|-------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------|---------------------|-------------|------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------|
| [Custom ModelServer](https://github.com/kserve/kserve/tree/master/python/kserve/kserve) | -- | v1, v2 | v2 | -- | -- | [Custom Model](custom/custom_model) |
| [LightGBM MLServer](https://mlserver.readthedocs.io/en/latest/runtimes/lightgbm.html) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v2 | v2 | v1.3.2 (MLServer) | 3 | [LightGBM Iris V2](./lightgbm) |
| [LightGBM ModelServer](https://github.com/kserve/kserve/tree/master/python/lgbserver) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v1, v2 | v2 | v0.11 (KServe) | 3 | [LightGBM Iris](./lightgbm) |
| [MLFlow ModelServer](https://docs.seldon.io/projects/seldon-core/en/latest/servers/mlflow.html) | [Saved MLFlow Model](https://www.mlflow.org/docs/latest/python_api/mlflow.sklearn.html#mlflow.sklearn.save_model) | v2 | v2 | v1.3.2 (MLServer) | 1 | [MLFLow wine-classifier](./mlflow) |
| [PMML ModelServer](https://github.com/kserve/kserve/tree/master/python/pmmlserver) | [PMML](http://dmg.org/pmml/v4-4-1/GeneralStructure.html) | v1, v2 | v2 | v0.11 (KServe) | 3, 4 ([PMML4.4.1](https://github.com/autodeployai/pypmml)) | [SKLearn PMML](./pmml) |
| [SKLearn MLServer](https://github.com/SeldonIO/MLServer) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v2 | v2 | v1.3.2 (MLServer) | 1 | [SKLearn Iris V2](./sklearn/v2) |
| [SKLearn ModelServer](https://github.com/kserve/kserve/tree/master/python/sklearnserver) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v1, v2 | v2 | v0.11 (KServe) | 1.3 | [SKLearn Iris](./sklearn/v2) |
| [TFServing](https://www.tensorflow.org/tfx/guide/serving) | [TensorFlow SavedModel](https://www.tensorflow.org/guide/saved_model) | v1 | *tensorflow | 2.6.2 ([TFServing Versions](https://github.com/tensorflow/serving/releases)) | 2 | [TensorFlow flower](./tensorflow) |
| [TorchServe](https://pytorch.org/serve/server.html) | [Eager Model/TorchScript](https://pytorch.org/docs/master/generated/torch.save.html) | v1, v2, *torchserve | *torchserve | 0.8.0 (TorchServe) | 2 | [TorchServe mnist](./torchserve) |
| [Triton Inference Server](https://github.com/triton-inference-server/server) | [TensorFlow,TorchScript,ONNX](https://github.com/triton-inference-server/server/blob/r21.09/docs/model_repository.md) | v2 | v2 | 23.05-py3 (Triton) | 8 (TensoRT), 1, 2 (TensorFlow), 2 (PyTorch), 2 (Triton) [Compatibility Matrix](https://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html) | [Torchscript cifar](triton/torchscript) |
| [XGBoost MLServer](https://github.com/SeldonIO/MLServer) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v2 | v2 | v1.3.2 (MLServer) | 1 | [XGBoost Iris V2](./xgboost) |
| [XGBoost ModelServer](https://github.com/kserve/kserve/tree/master/python/xgbserver) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v1, v2 | v2 | v0.11 (KServe) | 1 | [XGBoost Iris](./xgboost) |
| Model Serving Runtime | Exported model | HTTP | gRPC | Default Serving Runtime Version | Supported Framework (Major) Version(s) | Examples |
|-------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------|---------------------|-------------|------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------|
| [Custom ModelServer](https://github.com/kserve/kserve/tree/master/python/kserve/kserve) | -- | v1, v2 | v2 | -- | -- | [Custom Model](custom/custom_model/README.md) |
| [LightGBM MLServer](https://mlserver.readthedocs.io/en/latest/runtimes/lightgbm.html) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v2 | v2 | v1.3.2 (MLServer) | 3 | [LightGBM Iris V2](./lightgbm/README.md) |
| [LightGBM ModelServer](https://github.com/kserve/kserve/tree/master/python/lgbserver) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v1, v2 | v2 | v0.11 (KServe) | 3 | [LightGBM Iris](./lightgbm/README.md) |
| [MLFlow ModelServer](https://docs.seldon.io/projects/seldon-core/en/latest/servers/mlflow.html) | [Saved MLFlow Model](https://www.mlflow.org/docs/latest/python_api/mlflow.sklearn.html#mlflow.sklearn.save_model) | v2 | v2 | v1.3.2 (MLServer) | 1 | [MLFLow wine-classifier](./mlflow/v2/README.md) |
| [PMML ModelServer](https://github.com/kserve/kserve/tree/master/python/pmmlserver) | [PMML](http://dmg.org/pmml/v4-4-1/GeneralStructure.html) | v1, v2 | v2 | v0.11 (KServe) | 3, 4 ([PMML4.4.1](https://github.com/autodeployai/pypmml)) | [SKLearn PMML](./pmml/README.md) |
| [SKLearn MLServer](https://github.com/SeldonIO/MLServer) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v2 | v2 | v1.3.2 (MLServer) | 1 | [SKLearn Iris V2](./sklearn/v2/README.md) |
| [SKLearn ModelServer](https://github.com/kserve/kserve/tree/master/python/sklearnserver) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v1, v2 | v2 | v0.11 (KServe) | 1.3 | [SKLearn Iris](./sklearn/v2/README.md) |
| [TFServing](https://www.tensorflow.org/tfx/guide/serving) | [TensorFlow SavedModel](https://www.tensorflow.org/guide/saved_model) | v1 | *tensorflow | 2.6.2 ([TFServing Versions](https://github.com/tensorflow/serving/releases)) | 2 | [TensorFlow flower](./tensorflow/README.md) |
| [TorchServe](https://pytorch.org/serve/server.html) | [Eager Model/TorchScript](https://pytorch.org/docs/master/generated/torch.save.html) | v1, v2, *torchserve | *torchserve | 0.8.0 (TorchServe) | 2 | [TorchServe mnist](./torchserve/README.md) |
| [Triton Inference Server](https://github.com/triton-inference-server/server) | [TensorFlow,TorchScript,ONNX](https://github.com/triton-inference-server/server/blob/r21.09/docs/model_repository.md) | v2 | v2 | 23.05-py3 (Triton) | 8 (TensoRT), 1, 2 (TensorFlow), 2 (PyTorch), 2 (Triton) [Compatibility Matrix](https://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html) | [Torchscript cifar](triton/torchscript/README.md) |
| [XGBoost MLServer](https://github.com/SeldonIO/MLServer) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v2 | v2 | v1.3.2 (MLServer) | 1 | [XGBoost Iris V2](./xgboost/README.md) |
| [XGBoost ModelServer](https://github.com/kserve/kserve/tree/master/python/xgbserver) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v1, v2 | v2 | v0.11 (KServe) | 1 | [XGBoost Iris](./xgboost/README.md) |



Expand Down
58 changes: 58 additions & 0 deletions docs/python_runtime_api/docs/index.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# KServe Python Runtime API
KServe's python runtime API implements a standardized python model server API following [open inference protocol](https://github.com/kserve/open-inference-protocol).
It encapsulates data plane API definitions and storage retrieval for models.

It provides many functionalities, including among others:

* Implements the data plane API following open inference protocol.
* Provide extensible model server and model API.
* Allow customizing pre-processing, prediction and post-processing handlers.
* Readiness and liveness Handlers.

## Installation

KServe Python SDK can be installed by `pip` or `poetry`.

### pip install

```sh
pip install kserve
```

### Poetry

Checkout KServe GitHub repository and Install via [poetry](https://python-poetry.org/).

```sh
cd kserve/python/kserve
peotry install
```

## API Reference
::: kserve.model_server
::: kserve.model

## Storage API
The storage API is used by KServe `Storage Initializer` which supports the following cloud storage providers.

The storage package is optional and can be installed via
```sh
pip install kserve[storage]
```

* Google Cloud Storage with a prefix: "gs://"
* By default, it uses `GOOGLE_APPLICATION_CREDENTIALS` environment variable for user authentication.
* If `GOOGLE_APPLICATION_CREDENTIALS` is not provided, anonymous client will be used to download the artifacts.
* S3 Compatible Object Storage with a prefix "s3://"
* For static credentials it uses `S3_ENDPOINT`, `AWS_ACCESS_KEY_ID`, and `AWS_SECRET_ACCESS_KEY` environment variables for authentication.
* Azure Blob Storage with the format: `https://{$STORAGE_ACCOUNT_NAME}.blob.core.windows.net/{$CONTAINER}/{$PATH}`
* By default, it uses anonymous client to download the artifacts.
* For e.g. https://kfserving.blob.core.windows.net/triton/simple_string/
* Persistent Volume Claim (PVC) with the format `pvc://{$pvcname}/[path]`.
* The `pvcname` is the name of the PVC that contains the model.
* The `[path]` is the relative path to the model on the PVC.
* For e.g. `pvc://mypvcname/model/path/on/pvc`
* Generic URI over either `HTTP` prefixed with `http://` or `HTTPS` prefixed with `https://`.
For example:
* `https://<some_url>.com/model.joblib`
* `http://<some_url>.com/model.zip`
Loading

0 comments on commit d20e3a3

Please sign in to comment.