Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ModelRegistry custom storage intializer example #346

Merged
merged 2 commits into from
Mar 24, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 58 additions & 4 deletions docs/modelserving/storage/storagecontainers.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,18 +36,45 @@ In a _ClusterStorageContainer_ spec, you can specify container resource requests

If a storage URI is supported by two or more _ClusterStorageContainer_ CRs, there is no guarantee which one will be used. **Please make sure that the URI format is only supported by one ClusterStorageContainer CR**.

## Custom Protocol Example

If you would like to use a custom protocol `model-registry://`, for example, you can create a custom image and add a new ClusterStorageContainer CR like this:
If you would like to use a custom protocol `model-registry://`, for example, you can create a custom image and add a new `ClusterStorageContainer` CR to make it available to KServe.

```yaml
### Create the `Custom Storage Initializer` Image

The first step is to create a custom container image that will be injected into the KServe deployment, as init container, and that will be in charge to download the model.

The only requirement is that the `Entrypoint` of this container image should take (and properly manage) 2 positional arguments:
1. __Source URI__: identifies the `storageUri` set in the `InferenceService`
2. __Destination Path__: the location where the model should be stored, e.g., `/mnt/models`

!!! note
KServe controller will take care of properly injecting your container image and invoking it with those proper arguments.

A more concrete example can be found [here](https://github.com/lampajr/model-registry-storage-initializer), where the storage initializer query an existing `model registry` service in order to retrieve the original location of the model that the user requested to deploy.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a preference on the repo location but we definitely want to keep this doc concise.


### Create the `ClusterStorageContainer` CR

Once the Custom Storage Initializer image is ready, you just need to create a new `ClusterStorageContainer` CR to make it available in the cluster. You just need to provide 2 essential information:
1. The _container spec definition_, this is strictly dependent on your own custom storage initializer image.
2. The _supported uri formats_ for which your custom storage initializer should be injected, in this case just `model-registry://`.

=== "kubectl"
```bash
kubectl apply -f - <<EOF
apiVersion: "serving.kserve.io/v1alpha1"
kind: ClusterStorageContainer
metadata:
name: abc
spec:
container:
name: storage-initializer
image: abc/custom-storage-initializer:latest
image: abc/model-registry-storage-initializer:latest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we intended to show the kubeflow model registry example here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is not a tagged and published version of the kf/model-registry-initializer yet, so I think that for now we could keep this as generic by using abc/model-registry-storage-initializer:latest.

The source code of that initiliazer has been created here https://github.com/kubeflow/model-registry/tree/main/csi, as soon as an image is tagged we could change the image here as well, wdyt?

env:
- name: MODEL_REGISTRY_BASE_URL
value: "$MODEL_REGISTRY_SERVICE.model-registry.svc.cluster.local:$MODEL_REGISTRY_REST_PORT"
- name: MODEL_REGISTRY_SCHEME
value: "http"
resources:
requests:
memory: 100Mi
Expand All @@ -57,8 +84,35 @@ spec:
cpu: "1"
supportedUriFormats:
- prefix: model-registry://

EOF
```

### Deploy the Model with `InferenceService`

Create the `InferenceService` with the `model-registry` specific URI format.

=== "kubectl"
```bash
kubectl apply -n kserve-test -f - <<EOF
apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
name: "iris-model"
spec:
predictor:
model:
modelFormat:
name: sklearn
storageUri: "model-registry://iris/v1"
EOF
```

!!! note
The only assumption here is that the ML model you are going to deploy has been already registered in the Model Registry, more information can be found in the [kubeflow/model-registry](https://github.com/kubeflow/model-registry) repository.

In this specific example the `model-registry://iris/v1` model is referring to a registered model pointing to `gs://kfserving-examples/models/sklearn/1.0/model`. The crucial point here is that this information needs to be provided just during the registration process, whereas during every deployment action you just need to provide the `model-registry` specific URI that identifies that model (in this case `model-registry://${MODEL_NAME}/${MODEL_VERSION}`).

## Spec Attributes

Spec attributes are in [API Reference](/website/reference/api/#serving.kserve.io/v1alpha1.ClusterStorageContainer) doc.
Spec attributes are in [API Reference](/website/reference/api/#serving.kserve.io/v1alpha1.ClusterStorageContainer) doc.