-
Notifications
You must be signed in to change notification settings - Fork 127
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add ModelRegistry custom storage intializer example #346
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -36,18 +36,45 @@ In a _ClusterStorageContainer_ spec, you can specify container resource requests | |
|
||
If a storage URI is supported by two or more _ClusterStorageContainer_ CRs, there is no guarantee which one will be used. **Please make sure that the URI format is only supported by one ClusterStorageContainer CR**. | ||
|
||
## Custom Protocol Example | ||
|
||
If you would like to use a custom protocol `model-registry://`, for example, you can create a custom image and add a new ClusterStorageContainer CR like this: | ||
If you would like to use a custom protocol `model-registry://`, for example, you can create a custom image and add a new `ClusterStorageContainer` CR to make it available to KServe. | ||
|
||
```yaml | ||
### Create the `Custom Storage Initializer` Image | ||
|
||
The first step is to create a custom container image that will be injected into the KServe deployment, as init container, and that will be in charge to download the model. | ||
|
||
The only requirement is that the `Entrypoint` of this container image should take (and properly manage) 2 positional arguments: | ||
1. __Source URI__: identifies the `storageUri` set in the `InferenceService` | ||
2. __Destination Path__: the location where the model should be stored, e.g., `/mnt/models` | ||
|
||
!!! note | ||
KServe controller will take care of properly injecting your container image and invoking it with those proper arguments. | ||
|
||
A more concrete example can be found [here](https://github.com/lampajr/model-registry-storage-initializer), where the storage initializer query an existing `model registry` service in order to retrieve the original location of the model that the user requested to deploy. | ||
|
||
### Create the `ClusterStorageContainer` CR | ||
|
||
Once the Custom Storage Initializer image is ready, you just need to create a new `ClusterStorageContainer` CR to make it available in the cluster. You just need to provide 2 essential information: | ||
1. The _container spec definition_, this is strictly dependent on your own custom storage initializer image. | ||
2. The _supported uri formats_ for which your custom storage initializer should be injected, in this case just `model-registry://`. | ||
|
||
=== "kubectl" | ||
```bash | ||
kubectl apply -f - <<EOF | ||
apiVersion: "serving.kserve.io/v1alpha1" | ||
kind: ClusterStorageContainer | ||
metadata: | ||
name: abc | ||
spec: | ||
container: | ||
name: storage-initializer | ||
image: abc/custom-storage-initializer:latest | ||
image: abc/model-registry-storage-initializer:latest | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are we intended to show the kubeflow model registry example here? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There is not a tagged and published version of the kf/model-registry-initializer yet, so I think that for now we could keep this as generic by using The source code of that initiliazer has been created here https://github.com/kubeflow/model-registry/tree/main/csi, as soon as an image is tagged we could change the |
||
env: | ||
- name: MODEL_REGISTRY_BASE_URL | ||
value: "$MODEL_REGISTRY_SERVICE.model-registry.svc.cluster.local:$MODEL_REGISTRY_REST_PORT" | ||
- name: MODEL_REGISTRY_SCHEME | ||
value: "http" | ||
resources: | ||
requests: | ||
memory: 100Mi | ||
|
@@ -57,8 +84,35 @@ spec: | |
cpu: "1" | ||
supportedUriFormats: | ||
- prefix: model-registry:// | ||
|
||
EOF | ||
``` | ||
|
||
### Deploy the Model with `InferenceService` | ||
|
||
Create the `InferenceService` with the `model-registry` specific URI format. | ||
|
||
=== "kubectl" | ||
```bash | ||
kubectl apply -n kserve-test -f - <<EOF | ||
apiVersion: "serving.kserve.io/v1beta1" | ||
kind: "InferenceService" | ||
metadata: | ||
name: "iris-model" | ||
spec: | ||
predictor: | ||
model: | ||
modelFormat: | ||
name: sklearn | ||
storageUri: "model-registry://iris/v1" | ||
EOF | ||
``` | ||
|
||
!!! note | ||
The only assumption here is that the ML model you are going to deploy has been already registered in the Model Registry, more information can be found in the [kubeflow/model-registry](https://github.com/kubeflow/model-registry) repository. | ||
|
||
In this specific example the `model-registry://iris/v1` model is referring to a registered model pointing to `gs://kfserving-examples/models/sklearn/1.0/model`. The crucial point here is that this information needs to be provided just during the registration process, whereas during every deployment action you just need to provide the `model-registry` specific URI that identifies that model (in this case `model-registry://${MODEL_NAME}/${MODEL_VERSION}`). | ||
|
||
## Spec Attributes | ||
|
||
Spec attributes are in [API Reference](/website/reference/api/#serving.kserve.io/v1alpha1.ClusterStorageContainer) doc. | ||
Spec attributes are in [API Reference](/website/reference/api/#serving.kserve.io/v1alpha1.ClusterStorageContainer) doc. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't have a preference on the repo location but we definitely want to keep this doc concise.