Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: initial config #25

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
@@ -1 +1 @@
**/node_modules
**/node_modules
14 changes: 7 additions & 7 deletions .github/workflows/publish-autopilot-utils.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Publish Autopilot Utils to PyPI

on:
workflow_dispatch:

env:
PYTHON_APPS_FOLDER: yaku-apps-python

Expand All @@ -13,25 +13,25 @@ jobs:
- uses: actions/checkout@v4
with:
sparse-checkout: ${{ env.PYTHON_APPS_FOLDER }}

- name: Put all files from ${{ env.PYTHON_APPS_FOLDER }} in cwd
run: |
mv ${{ env.PYTHON_APPS_FOLDER }}/* .

- uses: actions/setup-python@v5
with:
python-version: "3.10"

- name: Setup pants
uses: pantsbuild/actions/init-pants@v8
with:
gha-cache-key: cache0
named-caches-hash: ${{ hashFiles('/3rdparty/*.txt') }}

- name: Build wheel
run: |
pants package ./packages/autopilot-utils/::

- name: Store the distribution packages
uses: actions/upload-artifact@v4
with:
Expand All @@ -57,4 +57,4 @@ jobs:
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
verify-metadata: false
verify-metadata: false
23 changes: 23 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.2.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
# Docker-specific hooks
- repo: https://github.com/hadolint/hadolint
rev: v2.10.0
hooks:
- id: hadolint # Lints Dockerfiles
# Conventional commit hooks for commit messages
- repo: https://github.com/compilerla/conventional-pre-commit
rev: v3.6.0
hooks:
- id: conventional-pre-commit
stages: [commit-msg]
args: []
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

!! THIS PROJECT IS UNDER CONSTRUCTION !!

<img src="./misc/under-construction.jpg" alt="Under Construction" width="300"/>
<img src="./misc/under-construction.jpg" alt="Under Construction" width="300"/>
<figcaption>
Foto from Mabel Amber from <a href="https://www.pexels.com/de-de/foto/nahaufnahme-fotografie-der-roten-und-weissen-strassenbeschilderung-117602/" target="_blank" style="color: #555;">Pexels</a>
</figcaption>
2 changes: 1 addition & 1 deletion chart/HELM_CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,4 +125,4 @@ When a new official version is created to be delivered to external customers, th


### Changed
- Updated db_cacert_secret_name to cacert_secret_name in sandbox-migration-job
- Updated db_cacert_secret_name to cacert_secret_name in sandbox-migration-job
2 changes: 1 addition & 1 deletion chart/README.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
# yaku-helm-chart
# yaku-helm-chart
2 changes: 1 addition & 1 deletion chart/charts/yaku-core-api/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ apiVersion: v2
name: yaku-core-api
description: Yaku core api
type: application
version: 0.1.0
version: 0.1.0
7 changes: 3 additions & 4 deletions chart/charts/yaku-core-api/templates/api-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ spec:
{{- with .Values.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- end }}
image: {{ .Values.global.images.yakuCoreApi }}
imagePullPolicy: {{ .Values.imagePullPolicy | default "IfNotPresent" }}
ports:
Expand Down Expand Up @@ -86,8 +86,8 @@ spec:
timeoutSeconds: {{ .Values.livenessProbe.timeoutSeconds }}
successThreshold: {{ .Values.livenessProbe.successThreshold }}
failureThreshold: {{ .Values.livenessProbe.failureThreshold }}
{{- end }}
env:
{{- end }}
env:
- name: NODE_TLS_REJECT_UNAUTHORIZED
value: {{ .Values.env.node_tls_reject_unauthorized | default "0" | quote }}
- name: PATH_PREFIX
Expand Down Expand Up @@ -330,4 +330,3 @@ spec:
path: cacert.crt
{{- end }}
{{- end }}

Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@ data:
{{ else }}
default_key: {{ index (lookup "v1" "Secret" .Release.Namespace "yaku-default-key").data "default_key" }}
{{ end }}
{{- end }}
{{- end }}
2 changes: 1 addition & 1 deletion chart/charts/yaku-core-api/templates/api-service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ spec:
{{- include "yaku.selectorLabels" . | nindent 4 }}
{{- with .Values.additionalSelectorLabels}}
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}
2 changes: 1 addition & 1 deletion chart/charts/yaku-core-api/templates/minio-secret.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ metadata:
data:
rootUser: bWluaW9hZG1pbg==
rootPassword: bWluaW9hZG1pbg==
{{- end }}
{{- end }}
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ data:

PGPASSWORD=$SANDBOX_DB_PASS pg_dump -d {{ .Values.env.db_name }} -h {{ .Values.global.database.db_host }} -U $SANDBOX_DB_USER --schema-only -O -x > /tmp/database-sandbox.db
cat /tmp/database-sandbox.db
{{- end }}
{{- end }}
Original file line number Diff line number Diff line change
Expand Up @@ -70,4 +70,4 @@ spec:
- key: {{ .Values.env.cacert_key_name | quote }}
path: postgres.crt
{{- end }}
{{- end }}
{{- end }}
4 changes: 2 additions & 2 deletions chart/charts/yaku-core-api/templates/yaku-postgres.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ spec:
- SELECT 1
initialDelaySeconds: 15
timeoutSeconds: 2
volumes:
volumes:
- name: pvc-data
persistentVolumeClaim:
claimName: {{ .Values.postgres.persistence.name }}
Expand All @@ -78,4 +78,4 @@ spec:
targetPort: 5432
selector:
app: {{ include "yaku.name" . }}-postgres-db
{{- end }}
{{- end }}
2 changes: 1 addition & 1 deletion chart/charts/yaku-core-api/values.yaml
Original file line number Diff line number Diff line change
@@ -1 +1 @@
# Default values for yaku-core-api
# Default values for yaku-core-api
2 changes: 1 addition & 1 deletion chart/charts/yaku-labs/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ apiVersion: v2
name: yaku-labs
description: Yaku Labs
type: application
version: 0.1.0
version: 0.1.0
2 changes: 1 addition & 1 deletion chart/charts/yaku-labs/templates/labs-configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ data:
probe.sh: |
#!/bin/sh
curl -X POST -d "{\"model\":\"llama3\"}" http://yaku-labs:11434/api/chat
{{- end }}
{{- end }}
6 changes: 3 additions & 3 deletions chart/charts/yaku-labs/templates/labs-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ metadata:
labels:
{{- include "yaku.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.replicaCount | default 1 }}
replicas: {{ .Values.replicaCount | default 1 }}
selector:
matchLabels:
{{- include "yaku.selectorLabels" . | nindent 6 }}
Expand Down Expand Up @@ -79,12 +79,12 @@ spec:
volumeMounts:
- name: probe-script
mountPath: /tmp/probe.sh
subPath: probe.sh
subPath: probe.sh
volumes:
- name: pvc-data
persistentVolumeClaim:
claimName: labs-pvc
- name: probe-script
configMap:
name: yaku-probe-config
{{- end }}
{{- end }}
1 change: 0 additions & 1 deletion chart/charts/yaku-labs/templates/labs-ingress.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,4 +59,3 @@ spec:
{{- end }}
{{- end }}
{{- end }}

2 changes: 1 addition & 1 deletion chart/charts/yaku-labs/templates/labs-service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@ spec:
{{- with .Values.additionalSelectorLabels}}
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}
{{- end }}
2 changes: 1 addition & 1 deletion chart/charts/yaku-labs/values.yaml
Original file line number Diff line number Diff line change
@@ -1 +1 @@
# Default values for yaku-ui
# Default values for yaku-ui
2 changes: 1 addition & 1 deletion chart/charts/yaku-ui/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ apiVersion: v2
name: yaku-ui
description: Yaku ui
type: application
version: 0.1.0
version: 0.1.0
2 changes: 1 addition & 1 deletion chart/charts/yaku-ui/templates/ui-configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ metadata:
data:
ui-config.json: |
{{ .Values.config.backendEnvironments | indent 4 -}}
{{- end }}
{{- end }}
4 changes: 2 additions & 2 deletions chart/charts/yaku-ui/templates/ui-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ metadata:
labels:
{{- include "yaku.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.replicaCount | default 1 }}
replicas: {{ .Values.replicaCount | default 1 }}
selector:
matchLabels:
{{- include "yaku.selectorLabels" . | nindent 6 }}
Expand Down Expand Up @@ -74,4 +74,4 @@ spec:
configMap:
name: {{ .Chart.Name }}
{{- end }}
{{- end }}
{{- end }}
1 change: 0 additions & 1 deletion chart/charts/yaku-ui/templates/ui-ingress.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,4 +59,3 @@ spec:
{{- end }}
{{- end }}
{{- end }}

2 changes: 1 addition & 1 deletion chart/charts/yaku-ui/templates/ui-service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@ spec:
{{- with .Values.additionalSelectorLabels}}
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}
{{- end }}
2 changes: 1 addition & 1 deletion chart/charts/yaku-ui/values.yaml
Original file line number Diff line number Diff line change
@@ -1 +1 @@
# Default values for yaku-ui
# Default values for yaku-ui
2 changes: 1 addition & 1 deletion chart/crds/argo-workflows-crds.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -871,4 +871,4 @@ spec:
- spec
type: object
served: true
storage: true
storage: true
12 changes: 6 additions & 6 deletions chart/documentation/configure-keycloak.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Based on your used Keycloak instance, please follow one of the option below.

1. Login with a user that have admin access to your realm `kcadm.sh config credentials --server <keycloak server> --realm <realm name> --user admin `

2. Run `./tools/setup-realm.sh`
2. Run `./tools/setup-realm.sh`

3. Go to `Client scopes`, change `global` Assigned type to `optional` and `namespace_1` Assigned type to `Default`

Expand All @@ -36,7 +36,7 @@ Based on your used Keycloak instance, please follow one of the option below.

Under (Choose the resources you want to import:), check: Clients, Realm roles and client roles.

Under (If a resource already exists, specify what should be done:) check: skip
Under (If a resource already exists, specify what should be done:) check: skip

You will then need to perform some manual steps to customize it and get the configuration working with your Yaku instance.

Expand All @@ -50,7 +50,7 @@ The pre-configured realm imported from [yaku-realm-configmap.yaml file](./tools/
A users in Keycloak is any entity that can login into your system. Yaku access is based on two users groups:
- **Admin users**

Admins have permissions to create and modify namespaces in a Yaku instance.
Admins have permissions to create and modify namespaces in a Yaku instance.
- **Namespace users**

Normal users that have access to one or multiple namespaces and can run Yaku assessments
Expand All @@ -61,7 +61,7 @@ After configuring your identity provider in Keycloak, users will be able to logi
### Configure Identity Provider

Keycloak acts as an Identity Broker to connect Yaku services with identity providers. It uses the provider’s identities to access Yaku services.
Keycloak bases identity providers on SAML v2.0, OpenID Connect v1.0 and OAuth v2.0 protocols. You can configure Keycloak to use any of these protocols to connect with your identity provider.
Keycloak bases identity providers on SAML v2.0, OpenID Connect v1.0 and OAuth v2.0 protocols. You can configure Keycloak to use any of these protocols to connect with your identity provider.

To add an identity provider, click on the `Identity Providers` tab in the left side menu and select the identity provider type you want to add. Keycloak displays the configuration page for the identity provider you selected.

Expand All @@ -70,7 +70,7 @@ We provide below two configuration examples for OpenID Connect and SAML.
#### OpenID Connect


This identity provider uses an Azure Entra ID OpenID Connect app registration created and configured in an Azure subscription.
This identity provider uses an Azure Entra ID OpenID Connect app registration created and configured in an Azure subscription.

1. Fill up the configuration page of the OpenID connect identity provider with the following values. You can find the `clientId` and `clientSecret` in your App registration details.

Expand Down Expand Up @@ -124,7 +124,7 @@ You can find more information about configuring OpenID Connect identity provider

#### SAML

This identity provider uses an Azure Entra ID SAML app registration created and configured in an Azure subscription.
This identity provider uses an Azure Entra ID SAML app registration created and configured in an Azure subscription.

1. Fill up the configuration page of the OpenID connect identity provider with the following values. You can find the `clientId` and `clientSecret` in your App registration details.

Expand Down
6 changes: 3 additions & 3 deletions chart/documentation/core-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ Yaku uses Argo Workflows workflow engine for orchestrating qg assessment jobs on

With Yaku helm chart, we ship Argo Workflows public helm chart as a sub chart. To get you started, we have configured Argo Workflows default values. In your production environment, you can tweak the Argo values based on your requirements.

A MinIO helm chart is also included as a sub chart to start Minio as an S3 artifact storage for Argo workflows.
A MinIO helm chart is also included as a sub chart to start Minio as an S3 artifact storage for Argo workflows.

Out of the box, the Minio deployment uses default credentials that are stored in a Kubernetes secret called **minio-creds**. To use different credentials, you need to create a new secret with your own credentials and add the secret name in the helm chart value _credentialsSecretName_. Read the variables documentation in the **values.yaml** file.

## Encryption Keys

**Important Notes**
**Important Notes**

The API has an internal mechanism to encrypt user sensitive data (yaku secrets) before storing them in tha database. You need to set up your own encryption key and create a Kubernetes secret that contains this key. The secret name and key should then be added in the following helm chart values:

Expand All @@ -37,7 +37,7 @@ The values in the helm chart should be:
- `encryption_secret_name: "my_yaku_encryption_secret"`
- `encryption_secret_key: "my_yaku_encryption_key"`

Out of the box, for trying Yaku in a playground setup, a predefined encryption key is generated that you can use. We recommend that you create your own value and deploy it as kuberenets secret, then add the kubernetes secret info in values file under `encryption_secret_name` and `encryption_secret_key`
Out of the box, for trying Yaku in a playground setup, a predefined encryption key is generated that you can use. We recommend that you create your own value and deploy it as kuberenets secret, then add the kubernetes secret info in values file under `encryption_secret_name` and `encryption_secret_key`

## Database

Expand Down
Loading