Updating subproperties after cluster creation is prohibited by kops cli

[shrugs]

kops doesn't accept adding the runtimeConfig with an existing cluster because the definition in config differs from cluster.spec (i.e., it thinks we're trying to set CloudProvider to "", and rejects the change).

The solution to get around this is to download config.spec and copy over the kubeAPIServer block over to config in full (and then add runtimeConfig).

After that I believe you can rolling-update as expected (untested).

[chrislovecnm]

@shrugs so we are missing pieces to the runtimeConfig section upon initial creation?

[shrugs]

@chrislovecnm I haven't looked at the code just yet, so this may be off, but the issue seems to be that when kops receives the updated config from kops edit cluster :cluster, it attempts to diff the changes with the existing cluster.spec. The process that diffs the old config with the new config seems to take non-specified values and coerce them into empty string. Then the process that diffs the new config and cluster.spec to determine which changes need to be applied to aws sees that we're attempting to change CloudProvider from "aws" to "", and denies all of the changes.

The workaround I posted above works because we're then trying to set CloudProvider from aws to aws, which is detected as no change, and passes the check.

I may have time later in the week to checkout the code and test a fix, but I don't want to make any guarantees.

[chrislovecnm]

@justinsb thoughts?

[brunoalano]

The solution by @shrugs works, I tested because I had the same error

[chrislovecnm]

We have PR inbound thay may help with this ... I am looking through issues to test against #1183 - no promises but we need to test.

[elblivion]

The workaround worked for me, but only after a rolling-update --force --yes as described in #618.

[shrugs]

I can confirm that this is no longer an issue on master, so I'm closing. In summary, after adding the additional properties (note the quotes around "true")

  kubeAPIServer:
    runtimeConfig:
      "batch/v2alpha1": "true"

you must do a forced rolling update (kops rolling-update cluster {cluster} --force --yes), since kops doesn't yet have a way to determine whether or not the process on the master is running with the correct arguments.

[snoby]

I can say that a fresh cluster version 1.5.4 then applying these changes the masters never come back online again after the rolling-update.