Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Integrate with Istio Ingress #588

Merged
merged 10 commits into from
May 31, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions doc/source/examples/istio_examples.nblink
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"path": "../../../notebooks/istio_example.ipynb"
}
1 change: 1 addition & 0 deletions doc/source/examples/notebooks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Notebooks
Go Model <go_example>
H2O Java MoJo <h2o_mojo>
Istio Canary <istio_canary>
Istio Examples <istio_examples>
Jaeger Tracing <tmpl_model_tracing>
Keras MNIST <keras_mnist>
Kubeflow Seldon E2E Pipeline <kubeflow_seldon_e2e_pipeline>
Expand Down
9 changes: 8 additions & 1 deletion doc/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,19 @@ Seldon Core is an open source platform for deploying machine learning models on
Annotation-based Configuration <graph/annotations.md>
Private Docker Registry <graph/private_registries.md>

.. toctree::
:maxdepth: 1
:caption: Ingress

Ambassador Ingress <ingress/ambassador.md>
Istio Ingress <ingress/istio.md>
Seldon OAuth Gateway <ingress/seldon.md>

.. toctree::
:maxdepth: 1
:caption: Deployment Options

Helm Charts <graph/helm_charts.md>
Ambassador Deployment <graph/ambassador.md>
Grafana Analytics <analytics/analytics.md>
Autoscaling <graph/autoscaling.md>

Expand Down
File renamed without changes.
51 changes: 51 additions & 0 deletions doc/source/ingress/istio.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Istio

Seldon core can be used in conjuction with [istio](https://istio.io/). Istio provides an [ingress gateway](https://istio.io/docs/tasks/traffic-management/ingress/) which Seldon Core can automatically wire up new deployments to. The steps to using istio are described below.

## Install Seldon Core Operator

Ensure when you install the seldon-core operator via Helm that you enabled istio. For example:

```bash
helm install seldon-core-operator --name seldon-core --set istio.enabled=true --repo https://storage.googleapis.com/seldon-charts --set usage_metrics.enabled=true
```

You need an istio gateway installed. By default we assume one called seldon-gateway. For example you can create this with the following yaml:

```
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: seldon-gateway
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
```

If you have your own gateway you will use then you can provide the name when installing the seldon operator. For example if your gateway is called `mygateway` you can install the operator with:

```bash
helm install seldon-core-operator --name seldon-core --set istio.enabled=true --set istio.gateway=mygateway --repo https://storage.googleapis.com/seldon-charts --set usage_metrics.enabled=true
```

You can also provide the gateway on a per Seldon Deployment resource basis by providing it with the annotation `seldon.io/istio-gateway`.

## Traffic Routing

Istio has the capability for fine grained traffic routing to your deployments. This allows:

* canary updates
* green-blue deployments
* A/B testing

An example showing canary updates can be found [here](../exmaples/istio_canary.html)



53 changes: 53 additions & 0 deletions doc/source/ingress/seldon.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Seldon OAuth Gateway

Seldon provides an example OAuth gateway you can use. We recommend however that you utilize for production gateway solutions such as Ambassador or istio.

## Install Seldon OAuth Gateway

You can install the Seldon OAuth gateway using Helm:

```bash
helm install helm-charts/seldon-core-oauth-gateway --name seldon-gateway --repo https://storage.googleapis.com/seldon-charts
```

## Provide OAuth Credentials

Provide OAuth credentials for your deployments when creating them. You should add `oauth_key` and `oauth_secret` values to your resource. For example:

```
apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
labels:
app: seldon
name: seldon-model
spec:
name: test-deployment
oauth_key: oauth-key
oauth_secret: oauth-secret
predictors:
- componentSpecs:
- spec:
containers:
- image: seldonio/mock_classifier:1.0
imagePullPolicy: IfNotPresent
name: classifier
resources:
requests:
memory: 1Mi
terminationGracePeriodSeconds: 1
graph:
children: []
endpoint:
type: REST
name: classifier
type: MODEL
labels:
version: v1
name: example
replicas: 1
```

## Serve Requests

To serve requests from your running deployment via the gateway follow the instructions [here](../workflow/serving.html#api-oauth-gateway)
4 changes: 2 additions & 2 deletions doc/source/python/python_module.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ The python package contains a module that provides a reference python client for
* Send data as tensor, TFTensor or ndarray
* External API
* Make REST or gRPC calls
* Call the API via Ambassador or Seldon's OAUTH API gateway.
* Call the API via Ambassador, Istio or Seldon's OAUTH API gateway.
* Test `predict` or `feedback` endpoints
* Provide a numpy array, binary data or string data as payload or get random data generated as payload for given shape
* Send data as tensor, TFTensor or ndarray
Expand All @@ -46,7 +46,7 @@ Basic usage of the client is to create a `SeldonClient` object first. For exampl

```python
from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="mymodel",namespace="seldon", ambassador_endpoint="localhost:8003")
sc = SeldonClient(deployment_name="mymodel",namespace="seldon", gateway_endpoint="localhost:8003")
```

Then make calls of various types. For example, to make a random prediction via the Ambassador gateway using REST:
Expand Down
16 changes: 14 additions & 2 deletions doc/source/workflow/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,19 +14,26 @@ First [install Helm](https://docs.helm.sh). When helm is installed you can deplo

```bash
helm install seldon-core-operator --name seldon-core --repo https://storage.googleapis.com/seldon-charts --set usage_metrics.enabled=true

```

Notes

* You can use ```--namespace``` to install the seldon-core controller to a particular namespace
* For full configuration options see [here](../reference/helm.md)

For particular ingresses we support you can inform the controller it should activate processing for them.

* Ambassador
* add `--set ambassador.enabled=true` : The controller will add annotations to services it creates so Ambassador can pick them up and wire an endpoint for your deployments.
* Istio Gateway
* add `--set istio.enabled=true` : The controller will create virtual services and destination rules to wire up endpoints in your istio ingress gateway.

## Install an Ingress Gateway

We presently support two API Ingress Gateways

* [Ambassador](https://www.getambassador.io/)
* [Istio Ingress](https://istio.io/)
* Seldon Core OAuth Gateway

### Install Ambassador
Expand All @@ -37,10 +44,15 @@ We suggest you install [the official helm chart](https://github.com/helm/charts/
helm install stable/ambassador --name ambassador --set crds.keep=false
```

### Install Istio Ingress Gateway

If you are using istio then the controller will create virtual services for an istio gateway. By default it will assume the gateway `seldon-gateway` as the name of the gateway. To change the default gateway add `--set istio.gateway=XYZ` when installing the seldon-core-operator.

### Install Seldon OAuth Gateway

This provides a basic OAuth Gateway.

```
```bash
helm install seldon-core-oauth-gateway --name seldon-gateway --repo https://storage.googleapis.com/seldon-charts
```

Expand Down
Loading