[BUG] Get DSA Key : You need to install \"jmespath\" prior to running json_query filter

[jdaln]

Hello!

Describe the bug
Running vagrant up results in

TASK [Get DSA keys] ************************************************************
fatal: [bullseye_vlan]: FAILED! => {"msg": "You need to install \"jmespath\" prior to running json_query filter"}

To Reproduce
Run vagrant up (Virtualbox, OSX)

Expected behavior
I expect this to run without errors.

System (lsb_release -a or similar):
Virtualbox, OSX
ansible-core 2.15.8
ansible-lint 6.22.2

Additional context
In fact, I was going to open a PR but my fix did not help.
I tried to add jmespath at the end of these lines in the Vagrant file.

ansible-role-hardening/Vagrantfile

Line 17 in e9b6226

inline: "apt-get update && apt-get -y install curl python3-pip && python3 -m pip install ansible"

like so:

      inline: "apt-get update && apt-get -y install curl python3-pip && python3 -m pip install ansible jmespath"

The interesting part is that it installs:

    bullseye_vlan: Successfully installed MarkupSafe-2.1.5 PyYAML-6.0.1 ansible-8.7.0 ansible-core-2.15.10 cffi-1.16.0 cryptography-42.0.5 importlib-resources-5.0.7 jinja2-3.1.3 jmespath-1.0.1 packaging-24.0 pycparser-2.22 resolvelib-1.0.1

[konstruktoid]

That should be installed at https://github.com/konstruktoid/ansible-role-hardening/blob/master/tests/debug_facts.yml#L2.
Can't reproduce with vagrant up bullseye_vlan

[konstruktoid]

PLAY RECAP *********************************************************************
bullseye_vlan              : ok=243  changed=131  unreachable=0    failed=0    skipped=139  rescued=0    ignored=0   

Friday 05 April 2024  11:36:48 +0000 (0:00:00.098)       0:03:52.041 ********** 
================================================
konstruktoid.hardening : Debian family package installation ------------ 36.89s
konstruktoid.hardening : Block blacklisted kernel modules -------------- 10.43s
konstruktoid.hardening : Enforce AppArmor profiles ---------------------- 9.06s
konstruktoid.hardening : Install rkhunter ------------------------------- 8.95s
konstruktoid.hardening : Add local usernames to password list ----------- 7.19s
konstruktoid.hardening : Run rkhunter propupd --------------------------- 6.72s
konstruktoid.hardening : Install rng-tools ------------------------------ 4.47s
konstruktoid.hardening : Install systemd timesyncd ---------------------- 4.19s
konstruktoid.hardening : Allow outgoing specified ports ----------------- 4.17s
konstruktoid.hardening : Update rkhunter databases ---------------------- 3.94s
konstruktoid.hardening : Set default deny ------------------------------- 3.93s
konstruktoid.hardening : Install python3-pexpect ------------------------ 3.76s
konstruktoid.hardening : Debian family UFW installation ----------------- 3.63s
konstruktoid.hardening : Blacklist misc kernel modules ------------------ 3.40s
konstruktoid.hardening : Install python3-passlib ------------------------ 2.79s
Install jmespath -------------------------------------------------------- 2.71s
konstruktoid.hardening : Configure apt ---------------------------------- 2.68s
konstruktoid.hardening : Ensure restrict compilers access via dpkg-statoverride --- 2.58s
konstruktoid.hardening : Restrict compilers access ---------------------- 2.55s
konstruktoid.hardening : Restart Postfix -------------------------------- 2.27s
Playbook run took 0 days, 0 hours, 3 minutes, 52 seconds

[jdaln]

I tried with vagrant up bullseye_vlan and I get the same error.
In https://github.com/konstruktoid/ansible-role-hardening/blob/master/tests/test.yml debug_facts.yml is declared after

    - name: Include role
      ansible.builtin.import_role:
        name: konstruktoid.hardening

[jdaln]

Do you perhaps have https://docs.ansible.com/ansible/latest/reference_appendices/config.html#ansible-pipelining setup somewhere in one of your ansible.cfg?

[jdaln]

Inverting konstruktoid.hardening and debug_facts does not work at all.

TASK [Install jmespath] ********************************************************
changed: [bullseye_vlan] => {"changed": true, "cmd": ["/usr/bin/python3", "-m", "pip.__main__", "install", "jmespath"], "name": ["jmespath"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Collecting jmespath\n  Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)\nInstalling collected packages: jmespath\nSuccessfully installed jmespath-1.0.1\n", "stdout_lines": ["Collecting jmespath", "  Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)", "Installing collected packages: jmespath", "Successfully installed jmespath-1.0.1"], "version": null, "virtualenv": null}

TASK [Debug "cpuinfo_rdrand handling, true"] ***********************************
fatal: [bullseye_vlan]: FAILED! => {"msg": "The conditional check 'ansible_local.cpuinfo.rdrand' failed. The error was: error while evaluating conditional (ansible_local.cpuinfo.rdrand): 'dict object' has no attribute 'cpuinfo'. 'dict object' has no attribute 'cpuinfo'\n\nThe error appears to be in '/Users/xxx/Dev/ansible-role-hardening-upstreeam/tests/debug_facts.yml': line 7, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Debug \"cpuinfo_rdrand handling, true\"\n  ^ here\n"}

Then I do not understand why it runs on your machine and not on mine

Edit:
I tried to write the test.yml as so but it does not help, same error:

---

- name: Include the konstruktoid.hardening role
  hosts: all
  any_errors_fatal: true
  tasks:
    - name: Install jmespath
      ansible.builtin.pip:
        name: jmespath
        state: present
    - name: Include role
      ansible.builtin.import_role:
        name: konstruktoid.hardening

- name: Debug facts
  hosts: all
  any_errors_fatal: true
  tasks:
    - name: Include debug tasks
      ansible.builtin.import_tasks:
        file: debug_facts.yml

I can run more tests on Monday, have a nice weekend!

[jdaln]

@konstruktoid I saw you have made commits over the weekend so I updated to the latest version but I still get

TASK [Get DSA keys] ************************************************************
fatal: [bullseye_vlan]: FAILED! => {"msg": "You need to install \"jmespath\" prior to running json_query filter"}

NO MORE HOSTS LEFT *************************************************************

PLAY RECAP *********************************************************************
bullseye_vlan              : ok=243  changed=131  unreachable=0    failed=1    skipped=136  rescued=0    ignored=0   

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

What I did:

• Updated to the latest repo
• ansible-galaxy install -r requirements.yml --force
• vagrant destroy
• vagrant up

I am still unsure how jmespath would be present before this step with the current code. What is strange is that adding it to the pip install earlier did not work. I will try with apt in the vagrant file next.

[konstruktoid]

~$ cat ~/.ansible.cfg 
[defaults]
callbacks_enabled = ansible.posix.profile_tasks,ansible.posix.timer
remote_tmp = /var/tmp/${USER}/ansible

[galaxy]
url = https://galaxy.ansible.com/

~$ ansible-galaxy install -r requirements.yml --force
[...]
~$ vagrant destroy -f
==> almalinux: VM not created. Moving on...
==> jammy: VM not created. Moving on...
==> focal: VM not created. Moving on...
==> bullseye: VM not created. Moving on...
==> bullseye_vlan: VM not created. Moving on...
~$ vagrant up bullseye_vlan
[...]
TASK [Install jmespath] ********************************************************
Monday 08 April 2024  08:09:56 +0000 (0:00:01.007)       0:03:47.191 ********** 
changed: [bullseye_vlan] => {"changed": true, "cmd": ["/usr/bin/python3", "-m", "pip.__main__", "install", "jmespath"], "name": ["jmespath"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Collecting jmespath\n  Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)\nInstalling collected packages: jmespath\nSuccessfully installed jmespath-1.0.1\n", "stdout_lines": ["Collecting jmespath", "  Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)", "Installing collected packages: jmespath", "Successfully installed jmespath-1.0.1"], "version": null, "virtualenv": null}
[...]
PLAY RECAP *********************************************************************
bullseye_vlan              : ok=243  changed=131  unreachable=0    failed=0    skipped=139  rescued=0    ignored=0   

Monday 08 April 2024  08:10:00 +0000 (0:00:00.097)       0:03:50.625 ********** 
==================================================
konstruktoid.hardening : Debian family package installation ------------ 36.31s
konstruktoid.hardening : Block blacklisted kernel modules -------------- 10.87s
konstruktoid.hardening : Enforce AppArmor profiles ---------------------- 9.19s
konstruktoid.hardening : Install rkhunter ------------------------------- 8.97s
konstruktoid.hardening : Add local usernames to password list ----------- 7.23s
konstruktoid.hardening : Run rkhunter propupd --------------------------- 6.70s
konstruktoid.hardening : Set default deny ------------------------------- 4.86s
konstruktoid.hardening : Install rng-tools ------------------------------ 4.61s
konstruktoid.hardening : Allow outgoing specified ports ----------------- 4.36s
konstruktoid.hardening : Install systemd timesyncd ---------------------- 4.15s  

[jdaln]

Sorry, I went through the log output a little better and, indeed, the install step runs. Now I understand how it works but very strangely it gives this:

TASK [Install jmespath] ********************************************************
ok: [bullseye_vlan] => {"changed": false, "cmd": ["/usr/bin/python3", "-m", "pip.__main__", "install", "jmespath"], "name": ["jmespath"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: jmespath in /usr/local/lib/python3.9/dist-packages (1.0.1)\n", "stdout_lines": ["Requirement already satisfied: jmespath in /usr/local/lib/python3.9/dist-packages (1.0.1)"], "version": null, "virtualenv": null}

and then later, this:

TASK [Get DSA keys] ************************************************************
fatal: [bullseye_vlan]: FAILED! => {"msg": "You need to install \"jmespath\" prior to running json_query filter"}

I will try with the same ansible.cfg but that should not make a difference. jmespath is not available in apt so I will keep testing other things.

[konstruktoid]

Just to verify, you do have jmespath installed on the host (running Vagrant)?
See https://github.com/konstruktoid/ansible-role-hardening/blob/master/requirements-dev.txt

[konstruktoid]

Updated the testing document, https://github.com/konstruktoid/ansible-role-hardening/blob/master/TESTING.md

[jdaln]

I did not have it installed. Sorry about that. Of course in this case it is the hosts' instance that it calls...

Going back to the TESTING.md.
If the goal is to get the users to have it running also with vagrant up, then currently, someone might bump into the following:

ERROR! the role 'konstruktoid.hardening' was not found in /Users/xxx/Dev/ansible-role-hardening-upstreeam/tests/roles:/Users/xxx/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/Users/ju4777/Dev/ansible-role-hardening-upstreeam/tests

The error appears to be in '/Users/xxx/Dev/ansible-role-hardening-upstreeam/tests/test.yml': line 8, column 15, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

      ansible.builtin.import_role:
        name: konstruktoid.hardening
              ^ here
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

On my side, I installed the latest master via ansible-galaxy.

[konstruktoid]

ERROR! the role 'konstruktoid.hardening' was not found in /Users/xxx/Dev/ansible-role-hardening-upstreeam/tests/roles:/Users/xxx/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/Users/ju4777/Dev/ansible-role-hardening-upstreeam/tests

https://github.com/konstruktoid/ansible-role-hardening?tab=readme-ov-file#requirements

And I've also update the test document, #601

[jdaln]

Thank you! Got everything running except a timeout in priviledge escalation for Jammy but I will try to run it again and if it reoccurs, I will attempt a merge request when I find the issue.