Upgrade procedure

[jovan-absci]

Hi guys,

What's the standard procedure for upgrading Kubeflow? I'm on version 1.3 and would like to bump it up to 1.5. I've installed it manually via manifests. I couldn't find any relevant documentation for this. Do I just run the kustomize commands for the 1.5 manifest? What about potentially obsolete namespaces (I assume they remain in the cluster)?

Any guidance would be appreciated, thanks.

[RakeshRaj97]

+1

[juliusvonkohout]

That is currently the task of the distributions and professional consulting.

[jovan-absci]

What if you're not using a distribution but the vanilla version you get from these manifests?
Is this intentionally obtuse so you need enterprise support to do it?

[missingcharacter]

@juliusvonkohout, what is the point for a member of https://github.com/kubeflow to make a comment that is not useful?

The question aims to help people understand how Kubeflow works and the planned upgrade path. If Kubeflow does not plan to support an upgrade path, I think it would be "OK" to say: "Create a new k8s cluster, install the new version of Kubeflow in this new cluster, and then migrate your workloads to this new Kubeflow deployment".

[juliusvonkohout]

@missingcharacter

"what is the point for a member of https://github.com/kubeflow to make a comment that is not useful?" I think my comment is quite useful, because it sets the baseline expectation for users. Currently that is the state and my comment is correct.

"Create a new k8s cluster, install the new version of Kubeflow in this new cluster, and then migrate your workloads to this new Kubeflow deployment" Is not an in-place upgrade, but just installing from scratch.

Users are looking for a way to do in place upgrades. I have some experience with such upgrades and it is possible, but there is no official way, especially no guarantee so far. So it is possible with advanced knowledge, but neither documented nor supported.

Nevertheless we are always looking for new volunteers to improve that situation. If you want to help, i am offering free mentoring for new Kubeflow contributors. Please reach out on Slack as well if you are interested on actually improving the situation.

@jovan-absci
"What if you're not using a distribution but the vanilla version you get from these manifests?
Is this intentionally obtuse so you need enterprise support to do it?"

It is not intentional, it is just the current state. And yes, it is good that companies such as Canonical, AWS, Google, IBM, freelancers etc. can make money of it, because in the end that develops and finances the open source project. +1 for the free market financing an open source project.

Nevertheless we are always looking for new volunteers to improve that situation. If you want to help, i am offering free mentoring for new Kubeflow contributors. You can spend your personal time on improving the situation for everyone using the vanilla manifests. Please reach out on Slack as well if you are interested on actually improving the situation.

we can continue the discussion in #2216

I am closing this since there are multiple issues regarding this.

/close

[google-oss-prow]

@juliusvonkohout: Closing this issue.

In response to this:

@missingcharacter

"what is the point for a member of https://github.com/kubeflow to make a comment that is not useful?" I think my comment is quite useful, because it sets the baseline expectation for users. Currently that is the state and my comment is correct.

"Create a new k8s cluster, install the new version of Kubeflow in this new cluster, and then migrate your workloads to this new Kubeflow deployment" Is not an in-place upgrade, but just installing from scratch.

Users are looking for a way to do in place upgrades. I have some experience with such upgrades and it is possible, but there is no official way, especially no guarantee so far. So it is possible with advanced knowledge, but neither documented nor supported.

Nevertheless we are always looking for new volunteers to improve that situation. If you want to help, i am offering free mentoring for new Kubeflow contributors. Please reach out on Slack as well if you are interested on actually improving the situation.

@jovan-absci
"What if you're not using a distribution but the vanilla version you get from these manifests?
Is this intentionally obtuse so you need enterprise support to do it?"

It is not intentional, it is just the current state. And yes, it is good that companies such as Canonical, AWS, Google, IBM, freelancers etc. can make money of it, because in the end that develops and finances the open source project. +1 for the free market financing an open source project.

Nevertheless we are always looking for new volunteers to improve that situation. If you want to help, i am offering free mentoring for new Kubeflow contributors. You can spend your personal time on improving the situation for everyone using the vanilla manifests. Please reach out on Slack as well if you are interested on actually improving the situation.

we can continue the discussion in #2216

I am closing this since there are multiple issues regarding this.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[missingcharacter]

@juliusvonkohout I made a couple of PRs and they are still open:

• Adding more examples #2103
• Allow adding extra headers to rstudio notebook kubeflow#6418

I do want to help.

[juliusvonkohout]

@missingcharacter amazing. We were sadly stalled due to the CNCF transition. Can you join the next manifests wg and notebooks wg meetings? You can reach me on slack as well as juliusvonkohout to discuss it further.

I just cleaned up from 140 to 52 issues and will do so with the PRs as well. The same goes for onboarding more first time contributors.

[juliusvonkohout]

and lets continue in #2216

[thesuperzapper]

I recommend checking out deployKF, as this kind of issue is going to be very difficult to fix upstream. This is because raw Kubeflow is distributed as a YAML manifests, so any customizations you make would have to be manually reapplied to new versions.

deployKF on the other hand uses a values config file (similar to helm), and the structure of these configs is forwards and backwards compatible, meaning that any customizations you make (for example, to connect your external identity provider or S3 bucket), won't have to be redone for new versions.

Similarly, if a resource is removed or renamed between versions, because deployKF uses ArgoCD Applications, you will be able to easily clean up resources that need to be "pruned" because they are no longer part of the new version.

Finally, uninstalling is as simple as just deleting those Argo CD applications (which will delete their child resources).

I'm interested to hear your feedback!

https://www.deploykf.org/