Move helm chart back to main repo (#2113)

* Move helm chart back to main repo Signed-off-by: pyalex <[email protected]> * clean up Signed-off-by: pyalex <[email protected]>
feast-dev · Dec 7, 2021 · 5826a9d · 5826a9d
1 parent 5d9e5f7
commit 5826a9d
Show file tree

Hide file tree

Showing 32 changed files with 1,096 additions and 47 deletions.
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -104,6 +104,7 @@ jobs:
     needs: get-version
     env:
       HELM_VERSION: v2.17.0
+      VERSION_WITHOUT_PREFIX: ${{ needs.get-version.outputs.version_without_prefix }}
     steps:
       - uses: actions/checkout@v2
       - uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
@@ -113,14 +114,16 @@ jobs:
           project_id: ${{ secrets.GCP_PROJECT_ID }}
           service_account_key: ${{ secrets.GCP_SA_KEY }}
       - run: gcloud auth configure-docker --quiet
-      - name: Validate chart release versions
-        run: ./infra/scripts/validate-helm-chart-docker-image.sh
       - name: Remove previous Helm
         run: sudo rm -rf $(which helm)
       - name: Install Helm
-        run: ./infra/scripts/install-helm.sh
+        run: ./scripts/helm/install-helm.sh
+      - name: Validate Helm chart prior to publishing
+        run: ./scripts/helm/validate-helm-chart-publish.sh
+      - name: Validate all version consistency
+        run: ./scripts/helm/validate-helm-chart-versions.sh $VERSION_WITHOUT_PREFIX
       - name: Publish Helm charts
-        run: ./infra/scripts/sync-helm-charts.sh
+        run: ./scripts/helm/push-helm-charts.sh $VERSION_WITHOUT_PREFIX
 
   publish-python-sdk:
     runs-on: ubuntu-latest

diff --git a/infra/charts/README.md b/infra/charts/README.md
diff --git a/infra/charts/feast/.helmignore b/infra/charts/feast/.helmignore
@@ -0,0 +1,22 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/
diff --git a/infra/charts/feast/Chart.yaml b/infra/charts/feast/Chart.yaml
@@ -0,0 +1,11 @@
+apiVersion: v1
+description: Feature store for machine learning
+name: feast
+version: 0.101.0
+keywords:
+  - machine learning
+  - big data
+  - mlops
+home: https://feast.dev/
+sources:
+  - https://github.com/feast-dev/feast
diff --git a/infra/charts/feast/README.md b/infra/charts/feast/README.md
@@ -0,0 +1,74 @@
+# Feast Helm Charts
+
+> :warning: **Disclaimer**: Since Feast 0.10 our vision is to manage all infrastructure for feature store from one place - Feast SDK. But while this new paradigm is still in development, we are planning to support the installation of some Feast components (like Java feature server) through Helm chart presented in this repository. However, we do not expect helm chart to become a long-term solution for deploying Feast components to production, and some frictions still might exist. For example, you will need to manually sync some configurations from [feature_store.yaml](https://docs.feast.dev/reference/feature-repository/feature-store-yaml) into the chart context (like path to the registry file or project name).
+
+This repo contains Helm charts for Feast components that are being installed on Kubernetes:
+* Feast (root chart): The complete Helm chart containing all Feast components and dependencies. Most users will use this chart, but can selectively enable/disable subcharts using the values.yaml file.
+    * [Feature Server](charts/feature-server): High performant JVM-based implementation of feature server.
+    * [Transformation Service](charts/transformation-service): Transformation server for calculating on-demand features
+    * Redis: (Optional) One of possible options for an online store used by Feature Server
+
+## Chart: Feast
+
+Feature store for machine learning Current chart version is `0.101.0`
+
+## Installation
+
+Charts are published to `https://feast-helm-charts.storage.googleapis.com`. Please note that this URL is different from the URL we previously used (`feast-charts`)
+
+Run the following commands to add the repository
+
+```
+helm repo add feast-charts https://feast-helm-charts.storage.googleapis.com
+helm repo update
+```
+
+Install Feast
+```
+helm install feast-release feast-charts/feast
+```
+
+## Customize your installation
+
+This Feast chart comes with a [values.yaml](values.yaml) that allows for configuration and customization of all sub-charts.
+
+In order to modify the default configuration of Feature Server, please use the `application-override.yaml` key in the `values.yaml` file in this chart. A code snippet example
+```
+feature-server:
+    application-override.yaml:
+        enabled: true
+        feast:
+            active_store: online
+            stores:
+            - name: online
+              type: REDIS
+              config:
+                host: localhost
+                port: 6379
+
+```
+
+For the default configuration, please see the [Feature Server Configuration](https://github.com/feast-dev/feast-java/blob/master/serving/src/main/resources/application.yml).
+
+For more details, please see: https://docs.feast.dev/how-to-guides/running-feast-in-production
+
+## Requirements
+
+| Repository | Name | Version |
+|------------|------|---------|
+| https://charts.helm.sh/stable | redis | 10.5.6 |
+| https://feast-helm-charts.storage.googleapis.com | feature-server(feature-server) | 0.101.0 |
+| https://feast-helm-charts.storage.googleapis.com | transformation-service(transformation-service) | 0.101.0 |
+
+## Values
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| feature-server.enabled | bool | `true` |  |
+| global.project | string | `"default"` | Project from feature_store.yaml |
+| global.registry | object | `{"cache_ttl_seconds":0,"path":"gs://path/to/registry.db"}` | Information about registry managed by Feast Python SDK (must be in sync with feature_store.yaml) |
+| global.registry.cache_ttl_seconds | int | `0` | Registry cache (in memory) will be refreshed on this interval |
+| global.registry.path | string | `"gs://path/to/registry.db"` | Path to the registry file managed by Feast Python SDK |
+| redis.enabled | bool | `false` | Flag to install Redis |
+| redis.usePassword | bool | `false` | Disable redis password |
+| transformation-service.enabled | bool | `true` |  |
diff --git a/infra/charts/feast/README.md.gotmpl b/infra/charts/feast/README.md.gotmpl
@@ -0,0 +1,57 @@
+# Feast Helm Charts
+
+> :warning: **Disclaimer**: Since Feast 0.10 our vision is to manage all infrastructure for feature store from one place - Feast SDK. But while this new paradigm is still in development, we are planning to support the installation of some Feast components (like Java feature server) through Helm chart presented in this repository. However, we do not expect helm chart to become a long-term solution for deploying Feast components to production, and some frictions still might exist. For example, you will need to manually sync some configurations from [feature_store.yaml](https://docs.feast.dev/reference/feature-repository/feature-store-yaml) into the chart context (like path to the registry file or project name).
+
+This repo contains Helm charts for Feast components that are being installed on Kubernetes:
+* Feast (root chart): The complete Helm chart containing all Feast components and dependencies. Most users will use this chart, but can selectively enable/disable subcharts using the values.yaml file.
+    * [Feature Server](charts/feature-server): High performant JVM-based implementation of feature server. 
+    * [Transformation Service](charts/transformation-service): Transformation server for calculating on-demand features
+    * Redis: (Optional) One of possible options for an online store used by Feature Server
+
+## Chart: Feast
+
+{{ template "chart.description" . }} Current chart version is `{{ template "chart.version" . }}`
+
+## Installation
+
+Charts are published to `https://feast-helm-charts.storage.googleapis.com`. Please note that this URL is different from the URL we previously used (`feast-charts`)
+
+Run the following commands to add the repository
+
+```
+helm repo add feast-charts https://feast-helm-charts.storage.googleapis.com
+helm repo update
+```
+
+Install Feast
+```
+helm install feast-release feast-charts/feast
+```
+
+## Customize your installation
+
+This Feast chart comes with a [values.yaml](values.yaml) that allows for configuration and customization of all sub-charts.
+
+In order to modify the default configuration of Feature Server, please use the `application-override.yaml` key in the `values.yaml` file in this chart. A code snippet example
+```
+feature-server:
+    application-override.yaml:
+        enabled: true
+        feast:
+            active_store: online
+            stores:
+            - name: online
+              type: REDIS
+              config:
+                host: localhost
+                port: 6379
+
+```
+
+For the default configuration, please see the [Feature Server Configuration](https://github.com/feast-dev/feast-java/blob/master/serving/src/main/resources/application.yml).
+
+For more details, please see: https://docs.feast.dev/how-to-guides/running-feast-in-production
+
+{{ template "chart.requirementsSection" . }}
+
+{{ template "chart.valuesSection" . }}
diff --git a/infra/charts/feast/charts/feature-server/Chart.yaml b/infra/charts/feast/charts/feature-server/Chart.yaml
@@ -0,0 +1,10 @@
+apiVersion: v1
+description: "Feast Feature Server: Online feature serving service for Feast"
+name: feature-server
+version: 0.100.4
+appVersion: v0.15.0
+keywords:
+- machine learning
+- big data
+- mlops
+home: https://github.com/feast-dev/feast
diff --git a/infra/charts/feast/charts/feature-server/README.md b/infra/charts/feast/charts/feature-server/README.md
@@ -0,0 +1,69 @@
+# feature-server
+
+![Version: 0.100.4](https://img.shields.io/badge/Version-0.100.4-informational?style=flat-square) ![AppVersion: v0.15.0](https://img.shields.io/badge/AppVersion-v0.15.0-informational?style=flat-square)
+
+Feast Feature Server: Online feature serving service for Feast
+
+**Homepage:** <https://github.com/feast-dev/feast>
+
+## Values
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| "application-generated.yaml".enabled | bool | `true` | Flag to include Helm generated configuration. Please set `application-override.yaml` to override this configuration. |
+| "application-override.yaml" | object | `{"enabled":true}` | Configuration to override the default [application.yaml](https://github.com/feast-dev/feast/blob/master/java/serving/src/main/resources/application.yml). Will be created as a ConfigMap. `application-override.yaml` has a higher precedence than `application-secret.yaml` |
+| "application-secret.yaml" | object | `{"enabled":true}` | Configuration to override the default [application.yaml](https://github.com/feast-dev/feast/blob/master/java/serving/src/main/resources/application.yml). Will be created as a Secret. `application-override.yaml` has a higher precedence than `application-secret.yaml`. It is recommended to either set `application-override.yaml` or `application-secret.yaml` only to simplify config management. |
+| "application.yaml".enabled | bool | `true` | Flag to include the default [configuration](https://github.com/feast-dev/feast/blob/master/java/serving/src/main/resources/application.yml). Please set `application-override.yaml` to override this configuration. |
+| envOverrides | object | `{}` | Extra environment variables to set |
+| image.pullPolicy | string | `"IfNotPresent"` | Image pull policy |
+| image.repository | string | `"feastdev/feature-server-java"` | Docker image for Feature Server repository |
+| image.tag | string | `"0.16.0"` | Image tag |
+| ingress.grpc.annotations | object | `{}` | Extra annotations for the ingress |
+| ingress.grpc.auth.enabled | bool | `false` | Flag to enable auth |
+| ingress.grpc.class | string | `"nginx"` | Which ingress controller to use |
+| ingress.grpc.enabled | bool | `false` | Flag to create an ingress resource for the service |
+| ingress.grpc.hosts | list | `[]` | List of hostnames to match when routing requests |
+| ingress.grpc.https.enabled | bool | `true` | Flag to enable HTTPS |
+| ingress.grpc.https.secretNames | object | `{}` | Map of hostname to TLS secret name |
+| ingress.grpc.whitelist | string | `""` | Allowed client IP source ranges |
+| ingress.http.annotations | object | `{}` | Extra annotations for the ingress |
+| ingress.http.auth.authUrl | string | `"http://auth-server.auth-ns.svc.cluster.local/auth"` | URL to an existing authentication service |
+| ingress.http.auth.enabled | bool | `false` | Flag to enable auth |
+| ingress.http.class | string | `"nginx"` | Which ingress controller to use |
+| ingress.http.enabled | bool | `false` | Flag to create an ingress resource for the service |
+| ingress.http.hosts | list | `[]` | List of hostnames to match when routing requests |
+| ingress.http.https.enabled | bool | `true` | Flag to enable HTTPS |
+| ingress.http.https.secretNames | object | `{}` | Map of hostname to TLS secret name |
+| ingress.http.whitelist | string | `""` | Allowed client IP source ranges |
+| javaOpts | string | `nil` | [JVM options](https://docs.oracle.com/cd/E22289_01/html/821-1274/configuring-the-default-jvm-and-java-arguments.html). For better performance, it is advised to set the min and max heap: <br> `-Xms2048m -Xmx2048m` |
+| livenessProbe.enabled | bool | `true` | Flag to enabled the probe |
+| livenessProbe.failureThreshold | int | `5` | Min consecutive failures for the probe to be considered failed |
+| livenessProbe.initialDelaySeconds | int | `60` | Delay before the probe is initiated |
+| livenessProbe.periodSeconds | int | `10` | How often to perform the probe |
+| livenessProbe.successThreshold | int | `1` | Min consecutive success for the probe to be considered successful |
+| livenessProbe.timeoutSeconds | int | `5` | When the probe times out |
+| logLevel | string | `"WARN"` | Default log level, use either one of `DEBUG`, `INFO`, `WARN` or `ERROR` |
+| logType | string | `"Console"` | Log format, either `JSON` or `Console` |
+| nodeSelector | object | `{}` | Node labels for pod assignment |
+| podLabels | object | `{}` | Labels to be added to Feast Serving pods |
+| readinessProbe.enabled | bool | `true` | Flag to enabled the probe |
+| readinessProbe.failureThreshold | int | `5` | Min consecutive failures for the probe to be considered failed |
+| readinessProbe.initialDelaySeconds | int | `15` | Delay before the probe is initiated |
+| readinessProbe.periodSeconds | int | `10` | How often to perform the probe |
+| readinessProbe.successThreshold | int | `1` | Min consecutive success for the probe to be considered successful |
+| readinessProbe.timeoutSeconds | int | `10` | When the probe times out |
+| replicaCount | int | `1` | Number of pods that will be created |
+| resources | object | `{}` | CPU/memory [resource requests/limit](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) |
+| secrets | list | `[]` | List of Kubernetes secrets to be mounted. These secrets will be mounted on /etc/secrets/<secret name>. |
+| service.grpc.nodePort | string | `nil` | Port number that each cluster node will listen to |
+| service.grpc.port | int | `6566` | Service port for GRPC requests |
+| service.grpc.targetPort | int | `6566` | Container port serving GRPC requests |
+| service.http.nodePort | string | `nil` | Port number that each cluster node will listen to |
+| service.http.port | int | `80` | Service port for HTTP requests |
+| service.http.targetPort | int | `8080` | Container port serving HTTP requests and Prometheus metrics |
+| service.type | string | `"ClusterIP"` | Kubernetes service type |
+| transformationService.host | string | `""` |  |
+| transformationService.port | int | `6566` |  |
+
+----------------------------------------------
+Autogenerated from chart metadata using [helm-docs v1.5.0](https://github.com/norwoodj/helm-docs/releases/v1.5.0)
diff --git a/infra/charts/feast/charts/feature-server/templates/_helpers.tpl b/infra/charts/feast/charts/feature-server/templates/_helpers.tpl
@@ -0,0 +1,45 @@
+{{/* vim: set filetype=mustache: */}}
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "feature-server.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "feature-server.fullname" -}}
+{{- if .Values.fullnameOverride -}}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
+{{- else -}}
+{{- $name := default .Chart.Name .Values.nameOverride -}}
+{{- if contains $name .Release.Name -}}
+{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
+{{- else -}}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+{{- end -}}
+{{- end -}}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "feature-server.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+
+{{/*
+Common labels
+*/}}
+{{- define "feature-server.labels" -}}
+app.kubernetes.io/name: {{ include "feature-server.name" . }}
+helm.sh/chart: {{ include "feature-server.chart" . }}
+app.kubernetes.io/instance: {{ .Release.Name }}
+{{- if .Chart.AppVersion }}
+app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
+{{- end }}
+app.kubernetes.io/managed-by: {{ .Release.Service }}
+{{- end -}}