Node taint k3s-controlplane=true:NoExecute

[Kerwood]

Version: v1.17.2+k3s1

Describe the bug
Well, not sure if its a bug.
I use k3sup for creating a cluster. By default, I don't want any work loads/pod scheduled on my master node other than what the master needs. For this i tried to use below parameter in the install command.

--node-taint k3s-controlplane=true:NoExecute

I kinda works, but it stops other kube-system pods at deploying as well.

NAMESPACE     NAME                                          READY   STATUS    RESTARTS   AGE
kube-system   pod/metrics-server-6d684c7b5-c8kmd            0/1     Pending   0          4m57s
kube-system   pod/local-path-provisioner-58fb86bdfd-fpxh5   0/1     Pending   0          4m57s
kube-system   pod/coredns-d798c9dd-nsg8f                    0/1     Pending   0          4m57s

Events:
  Type     Reason            Age        From               Message
  ----     ------            ----       ----               -------
  Warning  FailedScheduling  <unknown>  default-scheduler  0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate.

Expected behavior
I expected the master to function as normal, but not deploy new work loads/pods on the master node.

Not sure if there's a smarter way to achieve this.

---

Edit:

I found this in the docs.

By default, server nodes will be schedulable and thus your workloads can get launched on them. If you wish to have a dedicated control plane where no user workloads will run, you can use taints. The node-taint parameter will allow you to configure nodes with taints, for example --node-taint k3s-controlplane=true:NoExecute.

But it's preventing the above kube-system pods from deploying.

https://rancher.com/docs/k3s/latest/en/installation/ha/

[Kerwood]

My bad...

[Kerwood]

Nope.. That didn't fix it at all..

[Kerwood]

Here's my fix.. I just wish this was default on a fresh installation.

1.Taint the master with below command.

kubectl taint node dev-k3s-master k3s-controlplane=true:NoSchedule

2. Add tolerance on the control-plane services.

kubectl edit deployments local-path-provisioner -n kube-system

And add the following to the containers spec.

spec:
  ...
  template:
  ...
    spec:
    ...
      tolerations:
      - effect: NoExecute
        operator: Exists
      - effect: NoSchedule
        operator: Exists

Do the same for metrics-server and coredns. The latter will have will have tolerations: present, so just add the two effects to the list.

[FinalDoom]

Just a note for anyone who finds this, you don't have to modify any files (sticky config) as they already have tolerations, if you use the standard one:

kubectl taint node k3s-controller node-role.kubernetes.io/control-plane=true:NoSchedule