Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Render KServe Python Runtime API doc with mkdoc #333

Merged
merged 2 commits into from
Dec 31, 2023
Merged

Render KServe Python Runtime API doc with mkdoc #333

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b4c4231
Update KServe python sdk docs
yuzisun Dec 31, 2023
5da6652
Update serving runtime doc
yuzisun Dec 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions docs/modelserving/v1beta1/serving_runtime.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,20 +33,20 @@ These can also be found in the respective [runtime YAML](https://github.com/kser
For model frameworks using the KServe serving runtime, the specific default version can be found in [kserve/python](https://github.com/kserve/kserve/tree/master/python).
In a given serving runtime directory the pyproject.toml file contains the exact model framework version used. For example, in [kserve/python/lgbserver](https://github.com/kserve/kserve/tree/master/python/lgbserver) the [pyproject.toml](https://github.com/kserve/kserve/blob/master/python/lgbserver/pyproject.toml) file sets the model framework version to 3.3.2, `lightgbm ~= 3.3.2`.

| Model Serving Runtime | Exported model | HTTP | gRPC | Default Serving Runtime Version | Supported Framework (Major) Version(s) | Examples |
|-------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------|---------------------|-------------|------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------|
| [Custom ModelServer](https://github.com/kserve/kserve/tree/master/python/kserve/kserve) | -- | v1, v2 | v2 | -- | -- | [Custom Model](custom/custom_model) |
| [LightGBM MLServer](https://mlserver.readthedocs.io/en/latest/runtimes/lightgbm.html) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v2 | v2 | v1.3.2 (MLServer) | 3 | [LightGBM Iris V2](./lightgbm) |
| [LightGBM ModelServer](https://github.com/kserve/kserve/tree/master/python/lgbserver) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v1, v2 | v2 | v0.11 (KServe) | 3 | [LightGBM Iris](./lightgbm) |
| [MLFlow ModelServer](https://docs.seldon.io/projects/seldon-core/en/latest/servers/mlflow.html) | [Saved MLFlow Model](https://www.mlflow.org/docs/latest/python_api/mlflow.sklearn.html#mlflow.sklearn.save_model) | v2 | v2 | v1.3.2 (MLServer) | 1 | [MLFLow wine-classifier](./mlflow) |
| [PMML ModelServer](https://github.com/kserve/kserve/tree/master/python/pmmlserver) | [PMML](http://dmg.org/pmml/v4-4-1/GeneralStructure.html) | v1, v2 | v2 | v0.11 (KServe) | 3, 4 ([PMML4.4.1](https://github.com/autodeployai/pypmml)) | [SKLearn PMML](./pmml) |
| [SKLearn MLServer](https://github.com/SeldonIO/MLServer) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v2 | v2 | v1.3.2 (MLServer) | 1 | [SKLearn Iris V2](./sklearn/v2) |
| [SKLearn ModelServer](https://github.com/kserve/kserve/tree/master/python/sklearnserver) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v1, v2 | v2 | v0.11 (KServe) | 1.3 | [SKLearn Iris](./sklearn/v2) |
| [TFServing](https://www.tensorflow.org/tfx/guide/serving) | [TensorFlow SavedModel](https://www.tensorflow.org/guide/saved_model) | v1 | *tensorflow | 2.6.2 ([TFServing Versions](https://github.com/tensorflow/serving/releases)) | 2 | [TensorFlow flower](./tensorflow) |
| [TorchServe](https://pytorch.org/serve/server.html) | [Eager Model/TorchScript](https://pytorch.org/docs/master/generated/torch.save.html) | v1, v2, *torchserve | *torchserve | 0.8.0 (TorchServe) | 2 | [TorchServe mnist](./torchserve) |
| [Triton Inference Server](https://github.com/triton-inference-server/server) | [TensorFlow,TorchScript,ONNX](https://github.com/triton-inference-server/server/blob/r21.09/docs/model_repository.md) | v2 | v2 | 23.05-py3 (Triton) | 8 (TensoRT), 1, 2 (TensorFlow), 2 (PyTorch), 2 (Triton) [Compatibility Matrix](https://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html) | [Torchscript cifar](triton/torchscript) |
| [XGBoost MLServer](https://github.com/SeldonIO/MLServer) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v2 | v2 | v1.3.2 (MLServer) | 1 | [XGBoost Iris V2](./xgboost) |
| [XGBoost ModelServer](https://github.com/kserve/kserve/tree/master/python/xgbserver) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v1, v2 | v2 | v0.11 (KServe) | 1 | [XGBoost Iris](./xgboost) |
| Model Serving Runtime | Exported model | HTTP | gRPC | Default Serving Runtime Version | Supported Framework (Major) Version(s) | Examples |
|-------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------|---------------------|-------------|------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------|
| [Custom ModelServer](https://github.com/kserve/kserve/tree/master/python/kserve/kserve) | -- | v1, v2 | v2 | -- | -- | [Custom Model](custom/custom_model/README.md) |
| [LightGBM MLServer](https://mlserver.readthedocs.io/en/latest/runtimes/lightgbm.html) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v2 | v2 | v1.3.2 (MLServer) | 3 | [LightGBM Iris V2](./lightgbm/README.md) |
| [LightGBM ModelServer](https://github.com/kserve/kserve/tree/master/python/lgbserver) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v1, v2 | v2 | v0.11 (KServe) | 3 | [LightGBM Iris](./lightgbm/README.md) |
| [MLFlow ModelServer](https://docs.seldon.io/projects/seldon-core/en/latest/servers/mlflow.html) | [Saved MLFlow Model](https://www.mlflow.org/docs/latest/python_api/mlflow.sklearn.html#mlflow.sklearn.save_model) | v2 | v2 | v1.3.2 (MLServer) | 1 | [MLFLow wine-classifier](./mlflow/v2/README.md) |
| [PMML ModelServer](https://github.com/kserve/kserve/tree/master/python/pmmlserver) | [PMML](http://dmg.org/pmml/v4-4-1/GeneralStructure.html) | v1, v2 | v2 | v0.11 (KServe) | 3, 4 ([PMML4.4.1](https://github.com/autodeployai/pypmml)) | [SKLearn PMML](./pmml/README.md) |
| [SKLearn MLServer](https://github.com/SeldonIO/MLServer) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v2 | v2 | v1.3.2 (MLServer) | 1 | [SKLearn Iris V2](./sklearn/v2/README.md) |
| [SKLearn ModelServer](https://github.com/kserve/kserve/tree/master/python/sklearnserver) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v1, v2 | v2 | v0.11 (KServe) | 1.3 | [SKLearn Iris](./sklearn/v2/README.md) |
| [TFServing](https://www.tensorflow.org/tfx/guide/serving) | [TensorFlow SavedModel](https://www.tensorflow.org/guide/saved_model) | v1 | *tensorflow | 2.6.2 ([TFServing Versions](https://github.com/tensorflow/serving/releases)) | 2 | [TensorFlow flower](./tensorflow/README.md) |
| [TorchServe](https://pytorch.org/serve/server.html) | [Eager Model/TorchScript](https://pytorch.org/docs/master/generated/torch.save.html) | v1, v2, *torchserve | *torchserve | 0.8.0 (TorchServe) | 2 | [TorchServe mnist](./torchserve/README.md) |
| [Triton Inference Server](https://github.com/triton-inference-server/server) | [TensorFlow,TorchScript,ONNX](https://github.com/triton-inference-server/server/blob/r21.09/docs/model_repository.md) | v2 | v2 | 23.05-py3 (Triton) | 8 (TensoRT), 1, 2 (TensorFlow), 2 (PyTorch), 2 (Triton) [Compatibility Matrix](https://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html) | [Torchscript cifar](triton/torchscript/README.md) |
| [XGBoost MLServer](https://github.com/SeldonIO/MLServer) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v2 | v2 | v1.3.2 (MLServer) | 1 | [XGBoost Iris V2](./xgboost/README.md) |
| [XGBoost ModelServer](https://github.com/kserve/kserve/tree/master/python/xgbserver) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v1, v2 | v2 | v0.11 (KServe) | 1 | [XGBoost Iris](./xgboost/README.md) |



Expand Down
58 changes: 58 additions & 0 deletions docs/python_runtime_api/docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# KServe Python Runtime API
KServe's python runtime API implements a standardized python model server API following [open inference protocol](https://github.com/kserve/open-inference-protocol).
It encapsulates data plane API definitions and storage retrieval for models.

It provides many functionalities, including among others:

* Implements the data plane API following open inference protocol.
* Provide extensible model server and model API.
* Allow customizing pre-processing, prediction and post-processing handlers.
* Readiness and liveness Handlers.

## Installation

KServe Python SDK can be installed by `pip` or `poetry`.

### pip install

```sh
pip install kserve
```

### Poetry

Checkout KServe GitHub repository and Install via [poetry](https://python-poetry.org/).

```sh
cd kserve/python/kserve
peotry install
```

## API Reference
::: kserve.model_server
::: kserve.model

## Storage API
The storage API is used by KServe `Storage Initializer` which supports the following cloud storage providers.

The storage package is optional and can be installed via
```sh
pip install kserve[storage]
```

* Google Cloud Storage with a prefix: "gs://"
* By default, it uses `GOOGLE_APPLICATION_CREDENTIALS` environment variable for user authentication.
* If `GOOGLE_APPLICATION_CREDENTIALS` is not provided, anonymous client will be used to download the artifacts.
* S3 Compatible Object Storage with a prefix "s3://"
* For static credentials it uses `S3_ENDPOINT`, `AWS_ACCESS_KEY_ID`, and `AWS_SECRET_ACCESS_KEY` environment variables for authentication.
* Azure Blob Storage with the format: `https://{$STORAGE_ACCOUNT_NAME}.blob.core.windows.net/{$CONTAINER}/{$PATH}`
* By default, it uses anonymous client to download the artifacts.
* For e.g. https://kfserving.blob.core.windows.net/triton/simple_string/
* Persistent Volume Claim (PVC) with the format `pvc://{$pvcname}/[path]`.
* The `pvcname` is the name of the PVC that contains the model.
* The `[path]` is the relative path to the model on the PVC.
* For e.g. `pvc://mypvcname/model/path/on/pvc`
* Generic URI over either `HTTP` prefixed with `http://` or `HTTPS` prefixed with `https://`.
For example:
* `https://<some_url>.com/model.joblib`
* `http://<some_url>.com/model.zip`
Loading