how to configure cri-dockerd with calico

[cristicalin]

We pursued an implementation of cri-dockerd in kubespray kubernetes-sigs/kubespray#8623 , an ansible based kubernetes deployment tool.

In the linked PR we discovered that the cri-dockerd does not use the configured CNI resulting in incorrect pod addressing.

Failed CI runs:

• https://github.com/kubernetes-sigs/kubespray/pull/8623/checks?check_run_id=5547765685
• https://github.com/kubernetes-sigs/kubespray/pull/8623/checks?check_run_id=5547629216

The configured pod CIDR for the CI is 10.233.64.0/18 but the pods get IP addresses like 172.17.0.6 and 172.17.0.7 which are allocated from the default docker network.

I tried forcing the use of calico with setting the cri-dockerd command line like this:

ExecStart=/usr/local/bin/cri-dockerd --container-runtime-endpoint fd:// --cni-bin-dir=/opt/cni/bin --cni-cache-dir=/var/lib/cni --cni-conf-dir=/etc/cni/net.d --network-plugin=calico --pod-cidr=10.233.64.0/18

But I get the following error when starting cri-dockerd:

Mar 15 09:57:08 localhost cri-dockerd[48989]: time="2022-03-15T09:57:08Z" level=info msg="Using CNI configuration file /etc/cni/net.d/10-calico.conflist"
Mar 15 09:57:08 localhost cri-dockerd[48989]: time="2022-03-15T09:57:08Z" level=fatal msg="didn't find compatible CNI plugin with given settings &{HairpinMode:none NonMasqueradeCIDR:10.0.0.0/8 PluginName:calico PluginBinDirString:/opt/cni/bin PluginBinDirs:[/opt/cni/bin] PluginConfDir:/etc/cni/net.d PluginCacheDir:/var/lib/cni MTU:0}: network plugin \"calico\" not found"
Mar 15 09:57:08 localhost systemd[1]: cri-dockerd.service: Main process exited, code=exited, status=1/FAILURE
Mar 15 09:57:08 localhost systemd[1]: cri-dockerd.service: Failed with result 'exit-code'.
Mar 15 09:57:08 localhost systemd[1]: Failed to start CRI Interface for Docker Application Container Engine.
Mar 15 09:57:10 localhost systemd[1]: cri-dockerd.service: Scheduled restart job, restart counter is at 3.
Mar 15 09:57:10 localhost systemd[1]: Stopped CRI Interface for Docker Application Container Engine.
Mar 15 09:57:10 localhost systemd[1]: cri-dockerd.service: Start request repeated too quickly.
Mar 15 09:57:10 localhost systemd[1]: cri-dockerd.service: Failed with result 'exit-code'.
Mar 15 09:57:10 localhost systemd[1]: Failed to start CRI Interface for Docker Application Container Engine.

The test CNI configuration:

(venv) root@instance-1:~/kubespray# ls -l /opt/cni/bin/
total 163508
-rwxr-xr-x 1 root root  3990548 Mar 15 09:42 bandwidth
-rwsr-xr-x 1 root root 47026188 Mar 15 09:42 calico
-rwsr-xr-x 1 root root 47026188 Mar 15 09:42 calico-ipam
-rwxr-xr-x 1 root root  3357992 Mar 15 09:42 flannel
-rwxr-xr-x 1 root root  3402808 Mar 15 09:42 host-local
-rwsr-xr-x 1 root root 47026188 Mar 15 09:42 install
-rwxr-xr-x 1 root root  3472123 Mar 15 09:42 loopback
-rwxr-xr-x 1 root root  3924908 Mar 15 09:42 portmap
-rw-r--r-- 1 root root  4555575 Mar 15 09:42 tags.txt
-rwxr-xr-x 1 root root  3622648 Mar 15 09:42 tuning
(venv) root@instance-1:~/kubespray# ls -l /etc/cni/net.d/
total 12
-rw-r--r-- 1 root root  709 Mar 15 09:42 10-calico.conflist
-rw-r--r-- 1 root root  715 Mar 15 09:42 calico.conflist.template
-rw------- 1 root root 2824 Mar 15 09:42 calico-kubeconfig
(venv) root@instance-1:~/kubespray# cat /etc/cni/net.d/10-calico.conflist 
{
  "name": "cni0",
  "cniVersion":"0.3.1",
  "plugins":[
    {
      "datastore_type": "kubernetes",
      "nodename": "localhost",
      "type": "calico",
      "log_level": "info",
      "log_file_path": "/var/log/calico/cni/cni.log",
      "ipam": {
        "type": "calico-ipam",
        "assign_ipv4": "true",
        "ipv4_pools": ["10.233.64.0/18"]
      },
      "policy": {
        "type": "k8s"
      },
      "kubernetes": {
        "kubeconfig": "/etc/cni/net.d/calico-kubeconfig"
      }
    },
    {
      "type":"portmap",
      "capabilities": {
        "portMappings": true
      }
    },
    {
      "type":"bandwidth",
      "capabilities": {
        "bandwidth": true
      }
    }
  ]
}

[cristicalin]

I think I fixed this, the --network-plugin=calico should be --network-plugin=cni instead of the specific plugin.

Please document this in the README.md at least.

[zentavr]

@cristicalin - thank you for your help.