[stable/mysql] Update mysql user passwords on secrets change

[komljen]

If you don't specify MySQL root and other users passwords they are auto-generated. However, if you run the Helm upgrade the new auto-generated secrets will become invalid. With this fix, it will be possible to change user-specified password also on Helm upgrade.

[komljen]

/assign @scottrigby

[rjkernick]

/assign

[rjkernick]

Please resolve the merge conflicts

[rjkernick]

/ok-to-test

[komljen]

/retest

[rjkernick]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: komljen, rjkernick
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: scottrigby

Assign the PR to them by writing /assign @scottrigby in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Miouge1]

Thanks @komljen I was looking for this exact feature.

[unguiculus]

@viglesiasce Do we really want to do this? We have this problem with all charts that auto-generate passwords. IMHO you should always set a password explicitly for purposes other then simple testing. Alternatively, there's the --reuse-values flag for helm upgrade. Automatically changing passwords on upgrade is not something I would expect.

/hold

[komljen]

IMHO you should always set a password explicitly for purposes other then simple testing.

I agree on this one, but when I tried this chart for the first time I got confused why my apps started to fail after I did the upgrade of umbrella chart where mysql was a dependency. I think that Jenkins chart is changing the password on upgrade if not set also.

[Miouge1]

@unguiculus is --reuse-values really an alternative for this issue?

I have tested it like this:

helm upgrade --install mysql stable/mysql --set imageTag=5.7.10 --set persistence.enabled=false --reuse-values
kubectl exec $(kubectl get pods -l app=mysql-mysql -o jsonpath='{.items[0].metadata.name}') env | grep PASSWORD

helm upgrade --install mysql stable/mysql --set imageTag=5.7.11 --set persistence.enabled=false --reuse-values
kubectl exec $(kubectl get pods -l app=mysql-mysql -o jsonpath='{.items[0].metadata.name}') env | grep PASSWORD

which shows that passwords are re-randomized between both deploys. It would make sense since when the password is not set, the randAlphNum doesn't save its output to values no?

[viglesiasce]

Another approach would be to do the secret creation only on first install using the release number as a gate for example:

{{ if eq .Release.Revision 1 }}
# Secret YAML
{{ end }}

[komljen]

Good point, but if we specify our own password and then we want to change it, we could just use --set with helm upgrade and our MySQL would have a new password. Currently, we can not change the MySQL password with helm upgrade and this pull request solves that.

[scottrigby]

We have this problem with all charts that auto-generate passwords.

This has been on my mind for various charts - as well as those charts which depend on the ones auto-generating passwords. It would be good to agree on an approach across charts that have this problem. Can we address this at our next charts meeting? I have created #5167 to track this, as well as references a number of related issues/PRs/docs there.

[allanlewis]

Perhaps this justifies a "minor" rather than a "patch" version bump?

[komljen]

Yeah, this should be minor.

[cliedeman]

Created existingSecret option as an alternative solution - #5783

[komljen]

This pull request solves the problem of updating the password with helm upgrade command. It doesn't matter if the password is auto-generated or set by the user.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[stale]

This issue is being automatically closed due to inactivity.

[benlangfeld]

What prevented this from being merged?

[Tarjei400]

Bumping up @benlangfeld question, that would be really nice to have feature in a chart

[Tarjei400]

Just in case someone has same issue, it is possible to make exact this thing using extraContainers, extraVolumeMounts etc. below example how to parametrize a chart in values.yaml:

initializationFiles:
  mysql-init: |-
    UPDATE mysql.user SET authentication_string = PASSWORD('MYSQL_ROOT_PASSWORD')
    WHERE User = 'root' AND Host = '%';
    UPDATE mysql.user SET authentication_string = PASSWORD('MYSQL_PASSWORD')
    WHERE User = 'MYSQL_USER' AND Host = '%';
    FLUSH PRIVILEGES;

extraInitContainers: |
  - name: "update-mysql-init"
    image: "busybox:1.25.0"
    imagePullPolicy: {{ .Values.imagePullPolicy | quote }}
    command: [
      "sh",
      "-ce",
      "cp /docker-entrypoint-initdb.d/mysql-init /tmp/files/mysql-init &&
         sed -i \"s/MYSQL_ROOT_PASSWORD/$MYSQL_ROOT_PASSWORD/g\" /tmp/files/mysql-init &&
         sed -i \"s/MYSQL_PASSWORD/$MYSQL_PASSWORD/g\" /tmp/files/mysql-init &&
         sed -i \"s/MYSQL_USER/$MYSQL_USER/g\" /tmp/files/mysql-init" ]
    env:
      - name: MYSQL_ROOT_PASSWORD
        valueFrom:
          secretKeyRef:
            name: {{ template "mysql.fullname" . }}
            key: mysql-root-password
      - name: MYSQL_PASSWORD
        valueFrom:
          secretKeyRef:
            name: {{ template "mysql.fullname" . }}
            key: mysql-password
    volumeMounts:
      - name: migrations
        mountPath: /docker-entrypoint-initdb.d
      - name: tmp-files
        mountPath: /tmp/files

extraVolumes: |
  - name: tmp-files
    emptyDir: {}

extraVolumeMounts: |
  - name: tmp-files
    mountPath: /tmp/files```

[kylezs]

@Tarjei400 's solution did not work for me, perhaps because I was using a persistent volume.
Instead, I:

• Noted the current secrets being used for the mysql root and user (if you have a different one)
• Updated the secrets
• Restarted the pods (so the env vars updated)
• port-forwarded the mysql pod
• connected a local mysql client to the pod (using previously noted credentials of root user)
• used the mysql client to update the passwords
  ALTER USER 'userName'@'localhost' IDENTIFIED BY 'New-Password-Here';