README refers to non-existent file.

[Asday]

It is highly recommended to read deploy/claim.yaml to review available parameters and gain a better understanding of functionality and behavior.

I assume it's referring to deploy/class.yaml?

[travisghansen]

Yeah correct. If you’re just getting started please use this project: https://github.com/democratic-csi/charts

I haven’t fully documented it yet but the examples are pretty easy to follow/understand.

All non csi drivers are being phased out so this project is effectively in maintenance mode at this point.

[Asday]

I gotta be honest, I've got a FreeNAS home server, an ubuntu VM, and a tiny Django project, and I'm desperately lost in all of this. I've gotten as far as installing microk8s on my dev machine, and then falling into 1000 docs. While your link might not be being phased out, at least I can semi understand this from my complete beginner standpoint.

[travisghansen]

@Asday yeah, that's a lot to take in all at once for sure. Happy to help with this one or the other or just generally (been messing with k8s for several years at this point).

[Asday]

I think give it another weekend of looking at docs without writing a single line of YAML and I'll be ready to cry for help. I surely do appreciate it.

[travisghansen]

Yeah NP. I know the feeling and had many people help me so just 'paying it forward' :)

[haakonstorm]

I'm jumping on @Asday 's train here as an old sysadmin that just got aboard the k8s ship with a 5 x odroid n2 SBC cluster I'm setting up (for the 2nd time) which now finally shows what it hasn't previously:

NAME                              READY   STATUS    RESTARTS   AGE
pod/coredns-66bff467f8-rrvd9      1/1     Running   0          36m
pod/coredns-66bff467f8-snhlz      1/1     Running   0          36m
pod/etcd-o1                       1/1     Running   0          36m
pod/kube-apiserver-o1             1/1     Running   0          36m
pod/kube-controller-manager-o1    1/1     Running   0          36m
pod/kube-flannel-ds-arm64-8mhzb   1/1     Running   0          28m
pod/kube-flannel-ds-arm64-c7s69   1/1     Running   0          23m
pod/kube-flannel-ds-arm64-pf6pm   1/1     Running   0          23m
pod/kube-flannel-ds-arm64-qgznx   1/1     Running   0          24m
pod/kube-flannel-ds-arm64-wznbh   1/1     Running   0          23m
pod/kube-proxy-2p8qs              1/1     Running   0          24m
pod/kube-proxy-87vsr              1/1     Running   0          23m
pod/kube-proxy-df5nj              1/1     Running   0          23m
pod/kube-proxy-flnl8              1/1     Running   0          36m
pod/kube-proxy-mbr7p              1/1     Running   0          23m
pod/kube-scheduler-o1             1/1     Running   0          36m

NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE
service/kube-dns   ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP,9153/TCP   36m

NAME                                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/kube-flannel-ds-amd64     0         0         0       0            0           <none>                   28m
daemonset.apps/kube-flannel-ds-arm       0         0         0       0            0           <none>                   28m
daemonset.apps/kube-flannel-ds-arm64     5         5         5       5            5           <none>                   28m
daemonset.apps/kube-flannel-ds-ppc64le   0         0         0       0            0           <none>                   28m
daemonset.apps/kube-flannel-ds-s390x     0         0         0       0            0           <none>                   28m
daemonset.apps/kube-proxy                5         5         5       5            5           kubernetes.io/os=linux   36m

NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/coredns   2/2     2            2           36m

NAME                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/coredns-66bff467f8   2         2         2       36m

Oh all those glorious "Running"s.

Now, I need me some PersistentStorage, and my lovely FreeNas is set up with NFS and all, but I can't get any of that juicy storage provisioned on the cluster.

Looking at the content of https://github.com/democratic-csi/democratic-csi - is this something I should clone and deploy manually with kubectl -f or is this something I should give to helm?

I've added democratic-csi/democratic-csi to helm, and can get this:

λ helm show all democratic-csi/democratic-csi

apiVersion: v1
appVersion: "1.0"
description: A Helm chart for Kubernetes
name: democratic-csi
(...)

Any pointers where to go from here? O:-)

[nmaupu]

@haakonstorm
This is related to @travisghansen's democratic-csi, I suggest you open an issue there.

[travisghansen]

@haakonstorm lol that was some good writing! I hate to be the bearer of bad news but neither project will work for you ATM. You could pretty easily build your own image of this project for arm and run it I suppose though.

Unfortunately all the scaffolding for csi isn't multi-arch yet:

• Support multi-arch image kubernetes-csi/external-provisioner#381
• Add multiarch Dockerfile kubernetes-csi/external-attacher#224
• others

The csi driver itself is however built to be multi-arch at this point but not usable until everything else is as well.

[haakonstorm]

Thanks for the quick reply! The FreeNAS has an NFS share; and I really just need a simple way to provide persistent storage for my cluster. Any quick tips from the top of your head just to provide that share for the cluster to use?

[haakonstorm]

If that isn't viable, I'll look into compiling/building for arm64. I'm not familiar with the term csi apart from the crime series on TV, which isn't too shabby, but I'll go read some more bro-pages and see what comes up. Damn, k8s was harder than I thought. And on a totally unrelated note, the choice of arm64 is for the low-power reason, grab something to hold on to, because my unrelated project is getting my clustered powered by compressed air. I'll stop here.

[travisghansen]

LOL

If you're looking for really dirty you can literally mount the share(s) on the nodes using puppet/ansible/whatever (completely outside of k8s) and then use hostPath volumes.

• https://kubernetes.io/docs/concepts/storage/volumes/#hostpath

Less dirty is to 'manually' create the volumes (both in reality on the FreeNAS device and in k8s) and then create PVCs to consume those:

apiVersion: v1
kind: PersistentVolume
metadata:
  labels:
    foo: bar
  name: fancy_name
spec:
  mountOptions:
  - noatime
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 1Mi
  nfs:
    path: /some/nfs/path
    server: some_nfs_server
  persistentVolumeReclaimPolicy: Retain

# pvc snippet
  accessMode: ReadWriteMany
  size: 1Mi
  selector:
    matchLabels:
      foo: bar

Lastly you could attempt to use the nfs-client provisioner (there is a csi variant of this as well but again, no worky on arm, don't even know if they have an arm image for the original)

• https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client

If you you use that ^ all it does is literally create a new directory in a (presumably decently sized) share and all PVCs share the same...share. One downside is you won't get true volume sizes etc as all the PVCs/PVs will in reality be backed by a single share.

EDIT:

You could also run this project outside the cluster altogether which is entirely viable. See the development section of the README

• https://github.com/nmaupu/freenas-provisioner/blob/master/local-start.sh

[nmaupu]

There is also a local volume provisioner available from k8s 1.14 onward for some specific use cases:
https://kubernetes.io/docs/concepts/storage/volumes/#local

[travisghansen]

@haakonstorm apparently there are some non official images for all the csi bits for arm and someone has the csi driver running on arm..

democratic-csi/charts#5

[dbadavekc]

Thanks for the quick reply! The FreeNAS has an NFS share; and I really just need a simple way to provide persistent storage for my cluster. Any quick tips from the top of your head just to provide that share for the cluster to use?

There's the Helm nfs-client-provisioner available at https://kubernetes-charts.storage.googleapis.com/
I use it on my test sandbox cluster. Seems to work fine