NFS provisioner cannot handle "alpha" annotation for "default" storage class

[sputnick]

This affects all Kubernetes Helm Charts that default to using the storage class:

volume.alpha.kubernetes.io/storage-class: default

The logged error is:

Claim "default/wordypress-wordpress-wordpress": StorageClass "" not found

Changing the storage class key name to the "beta" version fixes things!?

volume.beta.kubernetes.io/storage-class: default

It would be great if the alpha version worked so that many many charts need not be changed and so backward compatibility is not lost. I'm using K8s 1.5.2 on a cluster built with kubeadm.

Relates to this ticket I created
helm/charts#807

[wongma7]

Background: The alpha annotation is a bit weird and predates this whole project. Basically if your PVC has the alpha annotation at all (set to anything, even blank) and if your cluster is on one of four(?) supported platforms (GCE, AWS, ?) kubernetes does some magic, using hardcoded parameters, and creates a PV for you. If your cluster isn't on one of those platforms, nothing happens. With the alpha annotation, there is actually no notion of storage classes, storage class objects didn't exist during the alpha period.

Beta behaves as you'd expect, it refers to a real storage class object and kubernetes looks at the storage class object to determine parameters for how to create a PV, instead of using hardcoded params.

nfs provisioner ignores the alpha annotation. If NFS provisioner were to respect the alpha annotation it would race with kubernetes when running on the special four platforms, which is not ideal. But alpha is alpha so I would be okay with making nfs provisioner do bad things like that if it improves UX and compatibility with charts.

Before I implement that I would like to hear what the helm folks have to say, IIRC there was already some buzz about the change in behavior between alpha & beta so they would know what the plans are. (Defaulting to volume.beta.kubernetes.io/storage-class: default won't make sense for everybody. Also there is this https://kubernetes.io/docs/user-guide/persistent-volumes/#class-1 "DefaultStorageClass admission plugin ")

[sputnick]

I've been discussing this on the #helm channel on slack today. People there are also not sure what the correct default approach should be but it certainly works to change the annotation to the "beta" key.

I created a class with the name "default" and also tried to setting
storageclass.beta.kubernetes.io/is-default-class : true
on another storage class and neither got picked by the NFS provisioner when the key was "alpha" like you said.

Perhaps you could detect if the cluster already has a default storage class or not?

I think we also need some more input from some of the Chart creators.

[sputnick]

My expectation I guess after looking at this from many angles is that at least

storageclass.beta.kubernetes.io/is-default-class : true

Should make this work whether it's alpha or beta?

[eparis]

many alpha annotations are intentionally broken when they move to beta. People should not be expecting alpha objects to keep working. The only way for us to realize that alpha means alpha is for alpha to stop working.

I know it is painful, but not supporting alpha pushes the entire ecosystem in the right direction, even though that direction really stinks.

[sputnick]

I totally agree with that. Need to get to the bottom of why all these charts are still using "alpha" ONLY for the use case of "default". They use beta for custom storage class names.

[jsafrane]

The current recommendation is at https://kubernetes.io/docs/user-guide/persistent-volumes/#writing-portable-configuration. Use alpha in your charts and expect users to have a default storage class. Or ask users for a class and fill it into beta annotation.

[hakamairi]

You could mark your storage class as default for example kubectl patch storageclass YOUR_STORAGE_CLASS_NAME -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

[wongma7]

closing; the linked https://kubernetes.io/docs/concepts/storage/persistent-volumes/#writing-portable-configuration + the info about defaultStorageClass should clear up any confusion. Hopefully soon the alpha annotation will be gone.