charts,salt: Bump NGINX Ingress chart to v4.10.3

Bump the NGINX Ingress chart to v4.10.3 and also bump the Ingress Controller image to v1.10.3 Upgrade NGINX Ingress chart using: ``` rm -rf charts/ingress-nginx helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm repo update helm fetch -d charts --untar ingress-nginx/ingress-nginx --version 4.10.3 ``` Re-render NGINX Ingress salt states using: ``` ./doit.sh codegen:chart_ingress-nginx ``` Also update the dashboards with ``` curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/helm-chart-4.10.3/deploy/grafana/dashboards/request-handling-performance.json \ -Lo salt/metalk8s/addons/nginx-ingress/deployed/files/ingress-nginx-performance.json curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/helm-chart-4.10.3/deploy/grafana/dashboards/request-handling-performance.json \ -Lo salt/metalk8s/addons/nginx-ingress/deployed/files/ingress-nginx-performance.json ```
scality · Aug 5, 2024 · c174d82 · c174d82
1 parent 9790dba
commit c174d82
Show file tree

Hide file tree

Showing 58 changed files with 2,192 additions and 971 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -6,6 +6,11 @@
 
 - Fix the log dashboard that was not showing any logs
   (PR[#4393](https://github.com/scality/metalk8s/pull/4393))
+- Bump ingress-nginx chart version to
+  [4.10.3](https://github.com/kubernetes/ingress-nginx/releases/tag/helm-chart-4.10.3)
+  The controller image has been bumped accordingly to
+  [v1.10.3](https://github.com/kubernetes/ingress-nginx/releases/tag/controller-v1.10.3)
+  (PR[#4395](https://github.com/scality/metalk8s/pull/4395))
 
 ## Release 127.0.4
 

diff --git a/buildchain/buildchain/versions.py b/buildchain/buildchain/versions.py
@@ -182,8 +182,8 @@ def _version_prefix(version: str, prefix: str = "v") -> str:
     ),
     Image(
         name="nginx-ingress-controller",
-        version="v1.9.4",
-        digest="sha256:5b161f051d017e55d358435f295f5e9a297e66158f136321d9b04520ec6c48a3",
+        version="v1.10.3",
+        digest="sha256:b5a5082f8e508cc1aac1c0ef101dc2f87b63d51598a5747d81d6cf6e7ba058fd",
     ),
     Image(
         name="node-exporter",

diff --git a/charts/ingress-nginx-control-plane.yaml b/charts/ingress-nginx-control-plane.yaml
@@ -3,7 +3,7 @@ controller:
 
   image:
     digest: null
-    repository: '{%- endraw -%}{{ build_image_name(\"nginx-ingress-controller\", False) }}{%- raw -%}'
+    repository: __image__(nginx-ingress-controller)
 
   electionID: ingress-control-plane-controller-leader
 

diff --git a/charts/ingress-nginx.yaml b/charts/ingress-nginx.yaml
@@ -3,7 +3,7 @@ controller:
 
   image:
     digest: null
-    repository: '{%- endraw -%}{{ build_image_name(\"nginx-ingress-controller\", False) }}{%- raw -%}'
+    repository: __image__(nginx-ingress-controller)
 
   hostPort:
     enabled: true

diff --git a/charts/ingress-nginx/Chart.yaml b/charts/ingress-nginx/Chart.yaml
@@ -1,21 +1,25 @@
 annotations:
-  artifacthub.io/changes: '- "Update Ingress-Nginx version controller-v1.9.4"'
+  artifacthub.io/changes: |
+    - Update Ingress-Nginx version controller-v1.10.3
   artifacthub.io/prerelease: "false"
 apiVersion: v2
-appVersion: 1.9.4
+appVersion: 1.10.3
 description: Ingress controller for Kubernetes using NGINX as a reverse proxy and
   load balancer
 home: https://github.com/kubernetes/ingress-nginx
 icon: https://upload.wikimedia.org/wikipedia/commons/thumb/c/c5/Nginx_logo.svg/500px-Nginx_logo.svg.png
 keywords:
 - ingress
 - nginx
-kubeVersion: '>=1.20.0-0'
+kubeVersion: '>=1.21.0-0'
 maintainers:
+- name: cpanato
+- name: Gacko
+- name: puerco
 - name: rikatz
 - name: strongjz
 - name: tao12345666333
 name: ingress-nginx
 sources:
 - https://github.com/kubernetes/ingress-nginx
-version: 4.8.4
+version: 4.10.3
diff --git a/charts/ingress-nginx/OWNERS b/charts/ingress-nginx/OWNERS
@@ -1,4 +1,4 @@
-# See the OWNERS docs: https://github.com/kubernetes/community/blob/master/contributors/guide/owners.md
+# See the OWNERS docs: https://www.kubernetes.dev/docs/guide/owners
 
 approvers:
 - ingress-nginx-helm-maintainers

diff --git a/charts/ingress-nginx/README.md b/charts/ingress-nginx/README.md
diff --git a/charts/ingress-nginx/changelog/helm-chart-4.10.0.md b/charts/ingress-nginx/changelog/helm-chart-4.10.0.md
@@ -0,0 +1,9 @@
+# Changelog
+
+This file documents all notable changes to [ingress-nginx](https://github.com/kubernetes/ingress-nginx) Helm Chart. The release numbering uses [semantic versioning](http://semver.org).
+
+### 4.10.0
+
+* - "Update Ingress-Nginx version controller-v1.10.0"
+
+**Full Changelog**: https://github.com/kubernetes/ingress-nginx/compare/helm-chart-4.9.1...helm-chart-4.10.0
diff --git a/charts/ingress-nginx/changelog/helm-chart-4.10.1.md b/charts/ingress-nginx/changelog/helm-chart-4.10.1.md
@@ -0,0 +1,11 @@
+# Changelog
+
+This file documents all notable changes to [ingress-nginx](https://github.com/kubernetes/ingress-nginx) Helm Chart. The release numbering uses [semantic versioning](http://semver.org).
+
+### 4.10.1
+
+* - "update post submit helm ci and clean up (#11221)"
+* - "refactor helm ci tests part I (#11188)"
+* - "Update Ingress-Nginx version controller-v1.10.1"
+
+**Full Changelog**: https://github.com/kubernetes/ingress-nginx/compare/helm-chart-4.10.0...helm-chart-4.10.1
diff --git a/charts/ingress-nginx/changelog/helm-chart-4.10.2.md b/charts/ingress-nginx/changelog/helm-chart-4.10.2.md
@@ -0,0 +1,18 @@
+# Changelog
+
+This file documents all notable changes to [ingress-nginx](https://github.com/kubernetes/ingress-nginx) Helm Chart. The release numbering uses [semantic versioning](http://semver.org).
+
+### 4.10.2
+
+* Chores: Align security contacts & chart maintainers to actual owners. (#11480)
+* Fix helm install on cloud provider admonition block (#11412)
+* edited helm-install tips (#11411)
+* added info for aws helm install (#11410)
+* add workflow to helm release and update ct for branch (#11317)
+* Merge pull request #11277 from strongjz/chart-1.10.1 (#11314)
+* release helm chart from release branch (#11278)
+* update post submit helm ci and clean up (#11221)
+* refactor helm ci tests part I (#11188)
+* Update Ingress-Nginx version controller-v1.10.2
+
+**Full Changelog**: https://github.com/kubernetes/ingress-nginx/compare/helm-chart-4.10.1...helm-chart-4.10.2
diff --git a/charts/ingress-nginx/changelog/helm-chart-4.10.3.md b/charts/ingress-nginx/changelog/helm-chart-4.10.3.md
@@ -0,0 +1,9 @@
+# Changelog
+
+This file documents all notable changes to [ingress-nginx](https://github.com/kubernetes/ingress-nginx) Helm Chart. The release numbering uses [semantic versioning](http://semver.org).
+
+### 4.10.3
+
+* Update Ingress-Nginx version controller-v1.10.3
+
+**Full Changelog**: https://github.com/kubernetes/ingress-nginx/compare/helm-chart-4.10.2...helm-chart-4.10.3
diff --git a/charts/ingress-nginx/changelog/helm-chart-4.9.0.md b/charts/ingress-nginx/changelog/helm-chart-4.9.0.md
@@ -0,0 +1,13 @@
+# Changelog
+
+This file documents all notable changes to [ingress-nginx](https://github.com/kubernetes/ingress-nginx) Helm Chart. The release numbering uses [semantic versioning](http://semver.org).
+
+### 4.9.0
+
+* - "Add controller.metrics.serviceMonitor.annotations in Helm chart"
+* - "fix(labels): use complete labels variable on default-backend deployment"
+* - "chart: allow setting allocateLoadBalancerNodePorts (#10693)"
+* - "[release-1.9] feat(helm): add documentation about metric args (#10695)"
+* - "Update Ingress-Nginx version controller-v1.9.5"
+
+**Full Changelog**: https://github.com/kubernetes/ingress-nginx/compare/helm-chart-4.8.3...helm-chart-4.9.0
diff --git a/charts/ingress-nginx/changelog/helm-chart-4.9.1.md b/charts/ingress-nginx/changelog/helm-chart-4.9.1.md
@@ -0,0 +1,10 @@
+# Changelog
+
+This file documents all notable changes to [ingress-nginx](https://github.com/kubernetes/ingress-nginx) Helm Chart. The release numbering uses [semantic versioning](http://semver.org).
+
+### 4.9.1
+
+* - "update web hook cert gen to latest release v20231226-1a7112e06"
+* - "Update Ingress-Nginx version controller-v1.9.6"
+
+**Full Changelog**: https://github.com/kubernetes/ingress-nginx/compare/helm-chart-4.9.0...helm-chart-4.9.1
diff --git a/charts/ingress-nginx/ci/daemonset-extra-modules.yaml b/charts/ingress-nginx/ci/daemonset-extra-modules.yaml
@@ -7,4 +7,7 @@ controller:
     type: ClusterIP
   extraModules:
     - name: opentelemetry
-      image: busybox
+      image:
+        registry: registry.k8s.io
+        image: busybox
+        tag: latest
diff --git a/charts/ingress-nginx/ci/deployment-extra-modules-default-container-sec-context.yaml b/charts/ingress-nginx/ci/deployment-extra-modules-default-container-sec-context.yaml
@@ -9,4 +9,7 @@ controller:
     allowPrivilegeEscalation: false
   extraModules:
     - name: opentelemetry
-      image: busybox
+      image:
+        registry: registry.k8s.io
+        image: busybox
+        tag: latest
diff --git a/charts/ingress-nginx/ci/deployment-extra-modules-specific-container-sec-context.yaml b/charts/ingress-nginx/ci/deployment-extra-modules-specific-container-sec-context.yaml
@@ -7,6 +7,9 @@ controller:
     type: ClusterIP
   extraModules:
     - name: opentelemetry
-      image: busybox
+      image:
+        registry: registry.k8s.io
+        image: busybox
+        tag: latest
       containerSecurityContext:
         allowPrivilegeEscalation: false
diff --git a/charts/ingress-nginx/ci/deployment-extra-modules.yaml b/charts/ingress-nginx/ci/deployment-extra-modules.yaml
@@ -7,4 +7,7 @@ controller:
     type: ClusterIP
   extraModules:
     - name: opentelemetry
-      image: busybox
+      image:
+        registry: registry.k8s.io
+        image: busybox
+        tag: latest
diff --git a/charts/ingress-nginx/ci/deployment-opentelemetry-customregistry-values.yaml b/charts/ingress-nginx/ci/deployment-opentelemetry-customregistry-values.yaml
@@ -0,0 +1,9 @@
+controller:
+  image:
+    repository: ingress-controller/controller
+    tag: 1.0.0-dev
+    digest: null
+  service:
+    type: ClusterIP
+  opentelemetry:
+    enabled: true
diff --git a/charts/ingress-nginx/templates/_helpers.tpl b/charts/ingress-nginx/templates/_helpers.tpl
@@ -242,22 +242,13 @@ Check the ingress controller version tag is at most three versions behind the la
 {{- end -}}
 {{- end -}}
 
-{{/*
-IngressClass parameters.
-*/}}
-{{- define "ingressClass.parameters" -}}
-  {{- if .Values.controller.ingressClassResource.parameters -}}
-          parameters:
-{{ toYaml .Values.controller.ingressClassResource.parameters | indent 4}}
-  {{ end }}
-{{- end -}}
-
 {{/*
 Extra modules.
 */}}
 {{- define "extraModules" -}}
 - name: {{ .name }}
-  image: {{ .image }}
+  {{- with .image }}
+  image: {{ if .repository }}{{ .repository }}{{ else }}{{ .registry }}/{{ .image }}{{ end }}:{{ .tag }}{{ if .digest }}@{{ .digest }}{{ end }}
   command:
   {{- if .distroless }}
     - /init_module
@@ -266,6 +257,7 @@ Extra modules.
     - -c
     - /usr/local/bin/init_module.sh
   {{- end }}
+  {{- end }}
   {{- if .containerSecurityContext }}
   securityContext: {{ toYaml .containerSecurityContext | nindent 4 }}
   {{- end }}

diff --git a/charts/ingress-nginx/templates/_params.tpl b/charts/ingress-nginx/templates/_params.tpl
@@ -29,7 +29,7 @@
 - --watch-namespace={{ default "$(POD_NAMESPACE)" .Values.controller.scope.namespace }}
 {{- end }}
 {{- if and (not .Values.controller.scope.enabled) .Values.controller.scope.namespaceSelector }}
-- --watch-namespace-selector={{ default "" .Values.controller.scope.namespaceSelector }}
+- --watch-namespace-selector={{ .Values.controller.scope.namespaceSelector }}
 {{- end }}
 {{- if and .Values.controller.reportNodeInternalIp .Values.controller.hostNetwork }}
 - --report-node-internal-ip-address={{ .Values.controller.reportNodeInternalIp }}
@@ -54,6 +54,9 @@
 {{- if .Values.controller.watchIngressWithoutClass }}
 - --watch-ingress-without-class=true
 {{- end }}
+{{- if not .Values.controller.metrics.enabled }}
+- --enable-metrics={{ .Values.controller.metrics.enabled }}
+{{- end }}
 {{- if .Values.controller.enableTopologyAwareRouting }}
 - --enable-topology-aware-routing=true
 {{- end }}

diff --git a/charts/ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml b/charts/ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml
@@ -43,7 +43,7 @@ spec:
       containers:
         - name: create
           {{- with .Values.controller.admissionWebhooks.patch.image }}
-          image: "{{- if .repository -}}{{ .repository }}{{ else }}{{ .registry }}/{{ .image }}{{- end -}}:{{ .tag }}{{- if (.digest) -}} @{{.digest}} {{- end -}}"
+          image: {{ if .repository }}{{ .repository }}{{ else }}{{ .registry }}/{{ .image }}{{ end }}:{{ .tag }}{{ if .digest }}@{{ .digest }}{{ end }}
           {{- end }}
           imagePullPolicy: {{ .Values.controller.admissionWebhooks.patch.image.pullPolicy }}
           args:

diff --git a/charts/ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml b/charts/ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml
@@ -43,7 +43,7 @@ spec:
       containers:
         - name: patch
           {{- with .Values.controller.admissionWebhooks.patch.image }}
-          image: "{{- if .repository -}}{{ .repository }}{{ else }}{{ .registry }}/{{ .image }}{{- end -}}:{{ .tag }}{{- if (.digest) -}} @{{.digest}} {{- end -}}"
+          image: {{ if .repository }}{{ .repository }}{{ else }}{{ .registry }}/{{ .image }}{{ end }}:{{ .tag }}{{ if .digest }}@{{ .digest }}{{ end }}
           {{- end }}
           imagePullPolicy: {{ .Values.controller.admissionWebhooks.patch.image.pullPolicy }}
           args:

diff --git a/charts/ingress-nginx/templates/controller-daemonset.yaml b/charts/ingress-nginx/templates/controller-daemonset.yaml
@@ -77,7 +77,7 @@ spec:
       containers:
         - name: {{ .Values.controller.containerName }}
           {{- with .Values.controller.image }}
-          image: "{{- if .repository -}}{{ .repository }}{{ else }}{{ .registry }}/{{ include "ingress-nginx.image" . }}{{- end -}}:{{ .tag }}{{ include "ingress-nginx.imageDigest" . }}"
+          image: {{ if .repository }}{{ .repository }}{{ else }}{{ .registry }}/{{ include "ingress-nginx.image" . }}{{ end }}:{{ .tag }}{{ include "ingress-nginx.imageDigest" . }}
           {{- end }}
           imagePullPolicy: {{ .Values.controller.image.pullPolicy }}
         {{- if .Values.controller.lifecycle }}
@@ -183,13 +183,13 @@ spec:
       {{- if .Values.controller.extraModules }}
         {{- range .Values.controller.extraModules }}
           {{- $containerSecurityContext := .containerSecurityContext | default $.Values.controller.containerSecurityContext }}
-          {{- include "extraModules" (dict "name" .name "image" .image "distroless" .distroless "containerSecurityContext" $containerSecurityContext "resources" .resources) | nindent 8 }}
+          {{- include "extraModules" (dict "name" .name "image" .image "containerSecurityContext" $containerSecurityContext "resources" .resources) | nindent 8 }}
         {{- end }}
       {{- end }}
       {{- if .Values.controller.opentelemetry.enabled }}
         {{- with .Values.controller.opentelemetry }}
           {{- $containerSecurityContext := .containerSecurityContext | default $.Values.controller.containerSecurityContext }}
-          {{- include "extraModules" (dict "name" .name "image" .image "distroless" .distroless "containerSecurityContext" $containerSecurityContext "resources" .resources) | nindent 8 }}
+          {{- include "extraModules" (dict "name" .name "image" .image "containerSecurityContext" $containerSecurityContext "resources" .resources) | nindent 8 }}
         {{- end }}
       {{- end }}
     {{- end }}

diff --git a/charts/ingress-nginx/templates/controller-deployment.yaml b/charts/ingress-nginx/templates/controller-deployment.yaml
@@ -19,7 +19,7 @@ spec:
     matchLabels:
       {{- include "ingress-nginx.selectorLabels" . | nindent 6 }}
       app.kubernetes.io/component: controller
-  {{- if not (or .Values.controller.autoscaling.enabled .Values.controller.keda.enabled) }}
+  {{- if eq .Values.controller.autoscaling.enabled .Values.controller.keda.enabled }}
   replicas: {{ .Values.controller.replicaCount }}
   {{- end }}
   revisionHistoryLimit: {{ .Values.revisionHistoryLimit }}
@@ -80,7 +80,7 @@ spec:
       containers:
         - name: {{ .Values.controller.containerName }}
           {{- with .Values.controller.image }}
-          image: "{{- if .repository -}}{{ .repository }}{{ else }}{{ .registry }}/{{ include "ingress-nginx.image" . }}{{- end -}}:{{ .tag }}{{ include "ingress-nginx.imageDigest" . }}"
+          image: {{ if .repository }}{{ .repository }}{{ else }}{{ .registry }}/{{ include "ingress-nginx.image" . }}{{ end }}:{{ .tag }}{{ include "ingress-nginx.imageDigest" . }}
           {{- end }}
           imagePullPolicy: {{ .Values.controller.image.pullPolicy }}
         {{- if .Values.controller.lifecycle }}
@@ -186,13 +186,13 @@ spec:
       {{- if .Values.controller.extraModules }}
         {{- range .Values.controller.extraModules }}
           {{- $containerSecurityContext := .containerSecurityContext | default $.Values.controller.containerSecurityContext }}
-          {{- include "extraModules" (dict "name" .name "image" .image "distroless" .distroless "containerSecurityContext" $containerSecurityContext "resources" .resources) | nindent 8 }}
+          {{- include "extraModules" (dict "name" .name "image" .image "containerSecurityContext" $containerSecurityContext "resources" .resources) | nindent 8 }}
         {{- end }}
       {{- end }}
       {{- if .Values.controller.opentelemetry.enabled }}
         {{- with .Values.controller.opentelemetry }}
           {{- $containerSecurityContext := .containerSecurityContext | default $.Values.controller.containerSecurityContext }}
-          {{- include "extraModules" (dict "name" .name "image" .image "distroless" .distroless "containerSecurityContext" $containerSecurityContext "resources" .resources) | nindent 8 }}
+          {{- include "extraModules" (dict "name" .name "image" .image "containerSecurityContext" $containerSecurityContext "resources" .resources) | nindent 8 }}
         {{- end }}
       {{- end }}
     {{- end }}

diff --git a/charts/ingress-nginx/templates/controller-ingressclass.yaml b/charts/ingress-nginx/templates/controller-ingressclass.yaml
@@ -1,6 +1,4 @@
 {{- if .Values.controller.ingressClassResource.enabled -}}
-# We don't support namespaced ingressClass yet
-# So a ClusterRole and a ClusterRoleBinding is required
 apiVersion: networking.k8s.io/v1
 kind: IngressClass
 metadata:
@@ -11,11 +9,13 @@ metadata:
     {{- toYaml . | nindent 4 }}
     {{- end }}
   name: {{ .Values.controller.ingressClassResource.name }}
-{{- if .Values.controller.ingressClassResource.default }}
+  {{- if .Values.controller.ingressClassResource.default }}
   annotations:
     ingressclass.kubernetes.io/is-default-class: "true"
-{{- end }}
+  {{- end }}
 spec:
   controller: {{ .Values.controller.ingressClassResource.controllerValue }}
-  {{ template "ingressClass.parameters" . }}
+  {{- with .Values.controller.ingressClassResource.parameters }}
+  parameters: {{ toYaml . | nindent 4 }}
+  {{- end }}
 {{- end }}
diff --git a/charts/ingress-nginx/templates/controller-keda.yaml b/charts/ingress-nginx/templates/controller-keda.yaml
@@ -1,4 +1,4 @@
-{{- if and .Values.controller.keda.enabled (eq .Values.controller.kind "Deployment") -}}
+{{- if and (eq .Values.controller.kind "Deployment") .Values.controller.keda.enabled (not .Values.controller.autoscaling.enabled) -}}
 apiVersion: {{ .Values.controller.keda.apiVersion }}
 kind: ScaledObject
 metadata:

diff --git a/charts/ingress-nginx/templates/controller-poddisruptionbudget.yaml b/charts/ingress-nginx/templates/controller-poddisruptionbudget.yaml
@@ -1,4 +1,13 @@
-{{- if or (and .Values.controller.autoscaling.enabled (gt (.Values.controller.autoscaling.minReplicas | int) 1)) (and (not .Values.controller.autoscaling.enabled) (gt (.Values.controller.replicaCount | int) 1)) }}
+# PDB is not supported for DaemonSets.
+# https://github.com/kubernetes/kubernetes/issues/108124
+{{- if eq .Values.controller.kind "Deployment" }}
+{{- $replicas := .Values.controller.replicaCount }}
+{{- if and .Values.controller.autoscaling.enabled (not .Values.controller.keda.enabled) }}
+{{- $replicas = .Values.controller.autoscaling.minReplicas }}
+{{- else if and .Values.controller.keda.enabled (not .Values.controller.autoscaling.enabled) }}
+{{- $replicas = .Values.controller.keda.minReplicas }}
+{{- end }}
+{{- if gt ($replicas | int) 1 }}
 apiVersion: {{ ternary "policy/v1" "policy/v1beta1" (semverCompare ">=1.21.0-0" .Capabilities.KubeVersion.Version) }}
 kind: PodDisruptionBudget
 metadata:
@@ -24,3 +33,4 @@ spec:
   maxUnavailable: {{ .Values.controller.maxUnavailable }}
   {{- end }}
 {{- end }}
+{{- end }}
diff --git a/charts/ingress-nginx/templates/controller-prometheusrules.yaml b/charts/ingress-nginx/templates/controller-prometheusrules.yaml
@@ -1,4 +1,4 @@
-{{- if and ( .Values.controller.metrics.enabled ) ( .Values.controller.metrics.prometheusRule.enabled ) ( .Capabilities.APIVersions.Has "monitoring.coreos.com/v1" ) -}}
+{{- if and .Values.controller.metrics.enabled .Values.controller.metrics.prometheusRule.enabled -}}
 apiVersion: monitoring.coreos.com/v1
 kind: PrometheusRule
 metadata: