How to install multiple instances of the provisioner in a cluster

[starypatyk]

It is not obvious how to install multiple instances of the provisioner in a cluster (e.g. pointing to different NFS servers or exported directories).

By trial and error I have found that there are two pre-requisites, at least when installing with Helm.

1. Different names must be used for StorageClasses. This is quite easy to figure out since the storageClassName attribute in the PVC is used to identify the provisioner.
2. Helm release names must be different - even when the charts are deployed into different namespaces. When trying to install a second instance of the provisioner with the same Helm release name I got the following error (conflict on ClusterRoles).

Error: rendered manifests contain a resource that already exists. Unable to continue with install: ClusterRole "nfs-prov-nfs-subdir-external-provisioner-runner" in namespace "" exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: key "meta.helm.sh/release-namespace" must equal "prod": current value is "dev"

Maybe it would make sense to provide an additional section in the documentation describing how to install multiple instances of the provisioner?

[Raskosk]

If you want to use multiple instances, you need to understand the following:

Seperating by addressed NFS-Servers (e.g. you want to address 2 different NFS-Servers for 2 different use-cases) -> separate deployment.yaml + separate StorageClass.yaml
Separating by lifecycle behaviour (e.g. some PVs should be transient/deleted after Pod is deleted and some other should be persistent) -> separate StorageClass.yaml

As far as I tested, you can reuse ClusterRole and ServiceAccount for multiple instances.

As your Error shows: you are trying to bring up multiple ClusterRoles of the same type with the same name.

I recommend starting with .yaml files and use "kustomize" to learn with.
Later, you can create your own helm chart on that files for your needs.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.