Helm: disabled PodSecurityPolicy in 1.25 still blocks upgrades

[dimitarvdimitrov]

The mimir-distributed helm chart doesn't render the PodSecurityPolicy resource when installed on clusters with version >= 1.25 (as of #2870)

mimir/operations/helm/charts/mimir-distributed/templates/_helpers.tpl

Lines 480 to 482 in e9bf061

	{{- define "mimir.rbac.usePodSecurityPolicy" -}}
	{{- and (semverCompare "< 1.25-0" (include "mimir.kubeVersion" .)) (and .Values.rbac.create (eq .Values.rbac.type "psp")) -}}
	{{- end -}}

Helm stores the history of manifests of resources in a kubernetes secret in the namespace of the release. So upon upgrading the 1.25 the helm chart no longer renders the manifest, however Helm still tries to recover it from the release history. This causes failed installations with the following error

resource mapping not found for name: "mimir" namespace: "" from "": no matches for kind "PodSecurityPolicy" in version "policy/v1beta1" ensure CRDs are installed first

Helm docs document this as expected behaviour (which effectively forbids using deprecated resources in the last release they are available)

Workarounds

• Delete the Kubernetes Secret that contains the release history. This forfeits the ability to do rollbacks.
• Upgrade the chart while still on kubernetes 1.24 but force omitting the PodSecurityPolicy with rbac.create: false

Proposed fix

Change the condition of omitting the PodSecurityContext resource to <= 1.24.

[dimitarvdimitrov]

This was reported in the community slack by Mario and Pea (thread; link expires after a few months)