k8s_drain not working properly with 'state: drain'

[felipe4334]

SUMMARY

when using module k8s_drain with state: drain it outputs an error of "local variable 'pod_names' referenced before assignment"

ISSUE TYPE

• Bug Report

COMPONENT NAME

kubernetes.core.k8s_drain with option: "state: drain"

ANSIBLE VERSION

2.11.6 and on 2.9

COLLECTION VERSION

~/.ansible/collections/ansible_collections : 2.2.1
~/../lib/python3.8/site-packages/ansible_collections: 1.2.1

CONFIGURATION

OS / ENVIRONMENT

Cent OS 7

STEPS TO REPRODUCE

EXPECTED RESULTS

Try draining a node with module like this:

    - name: drain the old nodes
      kubernetes.core.k8s_drain: 
        context: "{{ k8_cluster_context }}"
        state: drain  
        name: "{{ item }}"
        delete_options:
          terminate_grace_period: 900
      loop: "{{ k8_nodepool_privateips }}"
      register: k8_node_drain_results

ACTUAL RESULTS

failed: [localhost] (item=10.50.133.187) => {"ansible_loop_var": "item", "changed": false, "item": "10.50.133.187", "msg": "Failed to list pod from node 10.50.133.187 due to: local variable 'pod_names' referenced before assignment"}

[felipe4334]

state: cordon works as expected. Yes i did try remove state: drain as well and still same issue.

[Akasurde]

I'm investigating.

[Akasurde]

@gravesm Thanks for taking a look.

@felipe4334 Could you please check if PR #295 works for you and let us know? Thanks in advance.

[Akasurde]

resolved_by_pr #295

[TravisAbram]

I still have this issue with drain. (cordon/uncordon work as expected)

Collection Version: kubernetes.core 2.2.2

Error message

"msg": "Failed to list pod from node node_name due to: local variable 'pod_names' referenced before assignment"

Traceback shows an error on the same line

kubernetes.core/plugins/modules/k8s_drain.py

Line 186 in b19ff9d

pod_names

regardless of whether force is set to true or false.

Force: yes

      - name: Drain Node
        kubernetes.core.k8s_drain:
          name: "{{ inventory_hostname_short }}"
          state: drain
          delete_options:
            force: yes

The full traceback is:
  File "/tmp/ansible_kubernetes.core.k8s_drain_payload__uy247qv/ansible_kubernetes.core.k8s_drain_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s_drain.py", line 303, in delete_or_evict_pods
  File "/tmp/ansible_kubernetes.core.k8s_drain_payload__uy247qv/ansible_kubernetes.core.k8s_drain_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s_drain.py", line 186, in filter_pods
fatal: [node_fqdn]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "api_key": null,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "context": null,
            "delete_options": {
                "disable_eviction": false,
                "force": true,
                "ignore_daemonsets": false,
                "terminate_grace_period": null,
                "wait_sleep": 5,
                "wait_timeout": null
            },
            "host": null,
            "kubeconfig": null,
            "name": "node_name",
            "password": null,
            "persist_config": null,
            "proxy": null,
            "proxy_headers": null,
            "state": "drain",
            "username": null,
            "validate_certs": null
        }
    },
    "msg": "Failed to list pod from node node_name due to: local variable 'pod_names' referenced before assignment"
}

Force: no

      - name: Drain Node
        kubernetes.core.k8s_drain:
          name: "{{ inventory_hostname_short }}"
          state: drain
          delete_options:
            force: no

The full traceback is:
  File "/tmp/ansible_kubernetes.core.k8s_drain_payload_n2lqwj2t/ansible_kubernetes.core.k8s_drain_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s_drain.py", line 303, in delete_or_evict_pods
  File "/tmp/ansible_kubernetes.core.k8s_drain_payload_n2lqwj2t/ansible_kubernetes.core.k8s_drain_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s_drain.py", line 186, in filter_pods
fatal: [node_fqdn]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "api_key": null,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "context": null,
            "delete_options": {
                "disable_eviction": false,
                "force": false,
                "ignore_daemonsets": false,
                "terminate_grace_period": null,
                "wait_sleep": 5,
                "wait_timeout": null
            },
            "host": null,
            "kubeconfig": null,
            "name": "node_name",
            "password": null,
            "persist_config": null,
            "proxy": null,
            "proxy_headers": null,
            "state": "drain",
            "username": null,
            "validate_certs": null
        }
    },
    "msg": "Failed to list pod from node node_name due to: local variable 'pod_names' referenced before assignment"
}

[gravesm]

Hi @TravisAbram, the traceback suggests you are not actually using version 2.2.2. The first line of the traceback: line 303, in delete_or_evict_pods. This does not line up for version 2.2.2, but does for previous versions.

[jorneilander]

Hi @TravisAbram, the traceback suggests you are not actually using version 2.2.2. The first line of the traceback: line 303, in delete_or_evict_pods. This does not line up for version 2.2.2, but does for previous versions.

This is incorrect, in 2.2.2 this is most definitely still an issue:

kubernetes.core/plugins/modules/k8s_drain.py

Line 303 in d8538ff

pods, warnings, errors = filter_pods(pod_list.items, force, ignore_daemonset)

It is however, fixed in main:

kubernetes.core/plugins/modules/k8s_drain.py

Line 303 in b19ff9d

self._api_instance.create_namespaced_pod_eviction(

_{Reason for me saying this, is because I'm running into this exact issue...}

Should anyone run into the same issue:

ansible-galaxy collection install git+https://github.com/ansible-collections/kubernetes.core.git,main

This will install the latest version from main, which is still tagged 2.2.0 in Ansible Galaxy oddly enough.

[gravesm]

Sorry, you are correct. This change did not get backported to the 2.2 branch. I'll try to get a new 2.2 release out next week that includes this fix.