playbook fails when ShutdownGracePeriod and ShutdownGracePeriodCriticalPods are set

[lazybetrayer]

Environment:

• OS (printf "$(uname -srm)\n$(cat /etc/os-release)\n"):

Linux 3.10.0-514.26.2.el7.x86_64 x86_64
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

• Version of Ansible (ansible --version):

ansible 2.10.11
config file = /workspace/ansible.cfg
configured module search path = ['/workspace/library', '/workspace/kubespray/library']
ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
executable location = /usr/local/bin/ansible
python version = 3.8.5 (default, Jan 27 2021, 15:41:15) [GCC 9.3.0]

• Version of Python (python --version):
  Python 3.8.5

Kubespray version (commit) (git rev-parse --short HEAD): v2.17.0

Full inventory with variables (ansible -i inventory/sample/inventory.ini all -m debug -a "var=hostvars[inventory_hostname]"):

kubelet_shutdown_grace_period: 150s
kubelet_shutdown_grace_period_critical_pods: 30s

Output of ansible run:

fatal: [kube-node-10-200-17-47]: FAILED! => {
    "assertion": "kubelet_shutdown_grace_period > kubelet_shutdown_grace_period_critical_pods",
    "changed": false,
    "evaluated_to": false,
    "msg": "ShutdownGracePeriod (150s) needs to be greater than ShutdownGracePeriodCriticalPods (30s) in order to give normal pods time to be evacuated, please see https://kubernetes.io/blog/2021/04/21/grace
ful-node-shutdown-beta/ for details"
}

[champtar]

Set kubelet_shutdown_grace_period_critical_pods to 180s

[lazybetrayer]

Set kubelet_shutdown_grace_period_critical_pods to 180s

Still fails with ShutdownGracePeriod (150s) needs to be greater than ShutdownGracePeriodCriticalPods (180s).
I think kubelet_shutdown_grace_period and kubelet_shutdown_grace_period_critical_pods are compared as strings.

[champtar]

Was on my phone, this is kubespray assertion failing, would need to convert the duration to something we can compare, so I think we should just remove this assertion

[champtar]

@cristicalin #7746

[cristicalin]

The defaults are:

roles/kubespray-defaults/defaults/main.yaml:kubelet_shutdown_grace_period: 60s
roles/kubespray-defaults/defaults/main.yaml:kubelet_shutdown_grace_period_critical_pods: 20s

So it is kind of strange why the issue does not show up for the defaults. If indeed ansible does string comparison then that would explain it and we need to fix it.

A simple check would seem to confirm this:

(venv) root@ubuntu-nuc-00:~/kubespray# ansible localhost -m debug -e a=180s -e b=60s -a 'msg="{{ (a < b) | bool }}"'
[WARNING]: No inventory was parsed, only implicit localhost is available
localhost | SUCCESS => {
    "msg": true
}
(venv) root@ubuntu-nuc-00:~/kubespray# ansible localhost -m debug -e a=60s -e b=30s -a 'msg="{{ (a < b) | bool }}"'
[WARNING]: No inventory was parsed, only implicit localhost is available
localhost | SUCCESS => {
    "msg": false
}

I'll push a patch tomorrow as it's a bit late in the evening for me right now.

[cristicalin]

Turns out the fix is rather simple...