NXBT-3353: Use Jenkins Helm chart instead of Jenkins operator

- The operator currently has a big issue due to its backup mechanism: when deleting the Jenkins master pod while some builds are running, these builds get stuck after the new Jenkins master pod gets up and running. See jenkinsci/kubernetes-operator#542.
- The chart seems more "standard", relying on a StatefulSet.
- It's more complete: includes an Ingress template for instance.
- The plugin descriptor is standard, allowing for instance to not specify the version and always fetch the latest.
- It still provides CasC hot reload through a sidecar container.

Jenkinsfile

@@ -126,7 +126,7 @@ pipeline {
 
           echo """
           ----------------------------------------------------------------------
-          Synchronize K8s cluster state with Helmfile: Jenkins Operator, Jenkins
+          Synchronize K8s cluster state with Helmfile: Jenkins
           Namespace: ${NAMESPACE}
           ----------------------------------------------------------------------"""
           echo 'Current Helm 3 version:'

README.md

@@ -1,12 +1,12 @@
 # Nuxeo Platform CI
 
-Configuration as Code for the Platform CI.
+Configuration as Code for the Platform CI in Kubernetes.
 
 ## Jenkins
 
-We rely on the [Jenkins Operator](https://github.com/jenkinsci/kubernetes-operator) to manage Jenkins in Kubernetes.
+Jenkins is installed with the [Jenkins Helm chart](https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins).
 
-The operator is installed with [Helmfile](https://github.com/roboll/helmfile).
+The Helm chart is deployed with [Helmfile](https://github.com/roboll/helmfile).
 
 ### Requirements
 
@@ -17,61 +17,29 @@ The operator is installed with [Helmfile](https://github.com/roboll/helmfile).
 
 ### Installation
 
+#### Kubernetes Cluster Initialization
+
 Define the target namespace variable:
 
 ```shell
-NAMESPACE=targetnamespace
+NAMESPACE=target-namespace
 ```
 
-Install the Jenkins Custom Resource Definition:
+Create the `$NAMESPACE` namespace:
 
 ```shell
-kubectl apply -f https://raw.githubusercontent.com/jenkinsci/kubernetes-operator/master/deploy/crds/jenkins_v1alpha2_jenkins_crd.yaml
+kubectl create ns $NAMESPACE
 ```
 
-Patch it to allow synchronizing and patching the Jenkins custom resource with `helmfile`:
+Import the required secrets from the `platform` namespace:
 
 ```shell
-kubectl patch crd jenkins.jenkins.io --patch "$(NAMESPACE=$NAMESPACE envsubst < charts/jenkins-operator/patches/jenkins-crd.yaml)"
-kubectl patch crd jenkinsimages.jenkins.io --patch "$(NAMESPACE=$NAMESPACE envsubst < charts/jenkins-operator/patches/jenkins-crd.yaml)"
+./secrets/import-secrets.sh ./secrets/secrets platform
 ```
 
-Otherwise, we get the following error when running `helmfile sync`:
-
-> Error: UPGRADE FAILED: rendered manifests contain a resource that already exists. Unable to continue with update: CustomResourceDefinition "jenkins.jenkins.io" in namespace "" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "jenkins-operator"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "$NAMESPACE"
-
-Create the `$NAMESPACE` namespace and import the required secrets from the `platform-staging` or `platform` namespace:
+Create the secret containing the credentials for the Jenkins Configuration as Code (JCasC):
 
 ```shell
-kubectl create ns $NAMESPACE
-
-kubectl get secret platform-staging-tls --namespace=platform-staging --export -o yaml |\
-   kubectl apply --namespace=$NAMESPACE -f -
-
-kubectl get secret kubernetes-docker-cfg --namespace=platform-staging --export -o yaml |\
-   kubectl apply --namespace=$NAMESPACE -f -
-
-kubectl get secret jx-pipeline-git-github-git --namespace=platform-staging --export -o yaml |\
-   kubectl apply --namespace=$NAMESPACE -f -
-
-kubectl get secret kaniko-secret --namespace=platform-staging --export -o yaml |\
-   kubectl apply --namespace=$NAMESPACE -f -
-
-kubectl get secret packages.nuxeo.com-auth --namespace=platform-staging --export -o yaml |\
-   kubectl apply --namespace=$NAMESPACE -f -
-
-(
-cat << EOF
-apiVersion: v1
-kind: Secret
-type: Opaque
-metadata:
-  name: jenkins-docker-cfg
-data:
-  config.json: ********
-EOF
-) | kubectl apply --namespace=$NAMESPACE -f -
-
 (
 cat << EOF
 apiVersion: v1
@@ -80,34 +48,50 @@ type: Opaque
 metadata:
   name: jenkins-casc
 data:
-  GITHUB_OAUTH_CLIENT_ID: ********
-  GITHUB_OAUTH_SECRET: ********
-  GITHUB_USER: ********
-  GITHUB_TOKEN: ********
-  JIRA_PASSWORD: ********
-  SLACK_TOKEN: ********
+  gitHubOAuthClientId: ********
+  gitHubOAuthSecret: ********
+  gitHubUser: ********
+  gitHubToken: ********
+  jiraPassword: ********
+  slackToken: ********
 EOF
 ) | kubectl apply --namespace=$NAMESPACE -f -
+```
+
+Create the `ClusterRoleBinding` required for the `ServiceAccount` used by Jenkins, typically to create namespaces:
 
+```shell
 (
 cat << EOF
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRoleBinding
 metadata:
-  name: platform:jenkins-operator-master
+  name: $NAMESPACE:jenkins
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: cluster-admin
 subjects:
 - kind: ServiceAccount
-  name: jenkins-operator-master
+  name: jenkins
   namespace: $NAMESPACE
 EOF
 ) | kubectl apply -f -
 ```
 
-Sync all resources from the [state file](./helmfile.yaml):
+#### Kubernetes Cluster Synchronization
+
+Synchronize the Kubernetes cluster with the resources from the [helmfile](./helmfile.yaml).
+
+The following environment variables need to be set:
+
+- [packages.nuxeo.com](https://packages.nuxeo.com/) credentials:
+  - `PACKAGES_USERNAME`
+  - `PACKAGES_PASSWORD`
+
+- Nuxeo Connect credentials:
+  - `CONNECT_USERNAME`
+  - `CONNECT_PASSWORD`
 
 ```shell
 helmfile sync

charts/jenkins-extra/Chart.yaml

@@ -0,0 +1,4 @@
+apiVersion: v1
+name: jenkins-extra
+description: Jenkins extra templates
+version: 1.0.0

charts/jenkins-master/templates/secret-maven-settings.yaml → charts/jenkins-extra/templates/secret-maven-settings.yaml

@@ -4,7 +4,7 @@ type: Opaque
 metadata:
   name: jenkins-maven-settings
   labels:
-    app.kubernetes.io/name: jenkins-master
+    app.kubernetes.io/name: jenkins
     helm.sh/chart: {{ printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
@@ -15,4 +15,4 @@ metadata:
   annotations: {{ toYaml .Values.secret.mavenSettings.annotations | nindent 4 }}
   {{- end }}
 stringData:
-  settings.xml: | {{ .Values.secret.mavenSettings.xmlSettings | nindent 4 }}
+  settings.xml: | {{ .Values.secret.mavenSettings.xmlSettings | nindent 4 }}

charts/jenkins-master/values.yaml.gotmpl → charts/jenkins-extra/values.yaml.gotmpl

@@ -1,15 +1,3 @@
-ingress:
-  labels: {}
-  annotations:
-    kubernetes.io/ingress.class: nginx
-  service:
-    name: jenkins-operator-http-master
-    port: 8080
-  hostname: jenkins.{{ .Values.namespace }}.dev.nuxeo.com
-  tls:
-  - hosts:
-    - jenkins.{{ .Values.namespace }}.dev.nuxeo.com
-    secretName: {{ .Values.namespace }}-tls
 secret:
   mavenSettings:
     labels: {}
@@ -56,4 +44,4 @@ secret:
             </properties>
           </profile>
         </profiles>
-      </settings>
+      </settings>