Error generating reports with client_sw role

[cschar7]

Hi,

I downloaded the zip file and unzipped it on my RHEL 7 server. I am pretty much running the example playbook provided for the client_sw role against one host and I get errors when the generate_reports task runs for both the csv and html reports. Can you just use the client_sw_report.csv.j2 and client_sw_report.html.j2 templates as is from the zip file or do these require modifications?

I commented out the csv report and ran the playbook. Below is the error I see for the html report. Any ideas as to what causes this error?

TASK [roles/client_sw : generate reports] ***********************************************************************************************************************************
failed: [server1] (item={u'dest': u'client_sw_report.csv', u'src': u'client_sw_report.csv.j2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"dest": "client_sw_report.csv", "src": "client_sw_report.csv.j2"}, "msg": "AnsibleError: template error while templating string: expected token '=', got '.'. String: {# Print CSV header #}\nhostname,group,ip_address,os_distro,os_version,hw_arch,time,changed,unreachable,failed,details\n{# Loop through all hosts #}\n{% for host in ansible_play_hosts_all | sort %}\n{# System time #}\n{% if hostvars[host]['ansible_facts']['date_time'] is not defined %}\n{% set sys_time = '' %}\n{% else %}\n{% set sys_time = hostvars[host]['ansible_facts']['date_time']['date'] + ' ' + hostvars[host]['ansible_facts']['date_time']['time'] %}\n{% endif %}\n{# Package details #}\n{% set ns = namespace( details = [], changed = false ) %}\n{% for pkg_name in client_sw_pkg_state %}\n{% set pkg_file = hostvars[host]['ansible_facts']['sas_client_sw_pkgs']['packages'][pkg_name]['file'] | default('none', true) %}\n{% set pkg_vers = hostvars[host]['ansible_facts']['sas_client_sw_pkgs']['packages'][pkg_name]['vers'] | default('none', true) %}\n{% set pkg_req = hostvars[host]['ansible_facts']['sas_client_sw_' + pkg_name + 'req'] | default('none', true) %}\n{% set pkg_act = hostvars[host]['ansible_facts']['sas_client_sw' + pkg_name + 'act'] | default('none', true) %}\n{% set pkg_vers_beg = hostvars[host]['ansible_facts']['sas_client_sw' + pkg_name + 'vers_beg'] | default('none', true) %}\n{% set pkg_vers_end = hostvars[host]['ansible_facts']['sas_client_sw' + pkg_name + '_vers_end'] | default('none', true) %}\n{% set pkg_detail = {'key': pkg_name, 'value': {'installer_file': pkg_file, 'installer_version': pkg_vers, 'state_request': pkg_req, 'state_change': pkg_act, 'version_begin': pkg_vers_beg, 'version_end': pkg_vers_end}} %}\n{% if pkg_file != 'none' or pkg_act != 'none' %}\n{% set ns.details = ns.details + [pkg_detail] %}\n{% endif %}\n{% if pkg_act not in ['none', 'failed'] %}\n{% set ns.changed = true %}\n{% endif %}\n{% endfor %}\n{% set package_details = ns.details | items2dict %}\n{# Status #}\n{% set changed = ns.changed %}\n{% set unreachable = hostvars[host]['ansible_facts']['sas_client']['unreachable'] | default(False) %}\n{% set failed = hostvars[host]['ansible_facts']['sas_client']['failed'] | default(True) %}\n{% set msg = hostvars[host]['ansible_facts']['sas_client']['msg'] | default('Unexpected error occurred') %}\n{# Details #}\n{% set details = {\n 'hostname': host,\n 'msg': msg,\n 'unreachable': unreachable,\n 'failed': failed,\n 'changed': changed,\n 'packages': package_details\n }\n%}\n{# Format details #}\n{% if details %}\n{% if client_sw_reports_details_format | lower == 'json' %}\n{% set details = details | to_nice_json(indent=2) | replace("\"", "\"\"") %}\n{% else %}\n{% set details = details | to_nice_yaml(indent=2, width=160) | replace("\"", "\"\"") %}\n{% endif %}\n{% else %}\n{% set details = '' %}\n{% endif %}\n{# Print CSV line #}\n{{ '%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,"%s"' | format(\nhost,\nhostvars[host]['group_names'] | join(', '),\nhostvars[host]['ansible_facts']['default_ipv4']['address'] | default(),\nhostvars[host]['ansible_facts']['distribution'] | default(),\nhostvars[host]['ansible_facts']['distribution_version'] | default(),\nhostvars[host]['ansible_facts']['architecture'] | default(),\nsys_time,\nchanged,\nunreachable,\nfailed,\ndetails\n)}}\n{% endfor %}"}

Thanks,
Cullen

[llnagy76]

Hi Cullen,
Thank you for reporting this issue!
The csv.j2 and html.j2 template files should work without any changes.
Sadly, the error message (AnsibleError: template error while templating string: expected token '=', got '.') does not indicate the issue to me. Could you assist me with my investigation? Please rerun the example playbook, this time with the -v option and redirect the output to a file:
ansible-playbook -i inventory.yml run_client_sw.yml -v &> client_sw_result.txt
Then please attach the output file.
Thank you!
Laszlo

[cschar7]

Attached is the output with the -v option.

Thanks,
Cullen

client_sw_result.txt

[llnagy76]

Hi Cullen,
-v was insufficient to locate any serious traces. Please rerun the playbook, this time with the -vvv option and attach the output file.
ansible-playbook -i inventory.yml run_client_sw.yml -vvv &> client_sw_result_vvv.txt
Thanks,
Laszlo

[cschar7]

Output with the -vvv option

Thanks,
Cullen

client_sw_result.txt

[llnagy76]

Hi Cullen,
Even -vvv wasn't enough to figure out what is causing the problem. I have tried to modify my test system to be as similar to yours as possible but failed to reproduce the error. Let's try something else. Please run the following commands on the same computer where you run ansible:

$ python
>>> import jinja2
>>> jinja2.__version__

You are going to get something like this:

$ python
Python 2.7.18 (default, Sep 17 2021, 00:00:00) 
[GCC 10.3.1 20210422 (Red Hat 10.3.1-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import jinja2
>>> jinja2.__version__
'2.11.3'

Please share the commands' output. Thank you!

[cschar7]

[user1@server1 ~]$ python
Python 2.7.5 (default, Aug 13 2020, 02:51:10) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import jinja2
>>> jinja2.__version__
'2.7.2'

[llnagy76]

We need at least version 2.10. Is it possible for you to install a newer jinja2 version?

[cschar7]

I believe this version of jinja2 was installed when Red Hat Ansible Engine 2.9 was installed. I am checking with Red Hat support as I don't see a newer version of Ansible Engine in their repositories.

[cschar7]

I don't know if this approach works but I installed rh-python38.x86_64 and rh-python38-python-jinja2.noarch. Then I added "source scl_source enable rh-python38” to the .bashrc file for user1. After logging in as the user, this is what I get:

[user1@server1 ~]$ python --version
Python 3.8.11
[user1@server1 ~]$ python
Python 3.8.11 (default, Jul 23 2021, 14:55:16)
[GCC 9.1.1 20190605 (Red Hat 9.1.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.

import jinja2
jinja2.version
'2.10.3'

I ran the playbook again as user1 but still see the same error:

failed: [server1] (item={u'dest': u'client_sw_report.csv', u'src': u'client_sw_report.csv.j2'}) => {
"ansible_loop_var": "item",
"changed": false,
"item": {
"dest": "client_sw_report.csv",
"src": "client_sw_report.csv.j2"
},
"msg": "AnsibleError: template error while templating string: expected token '=', got '.'. String: {# Print CSV header #}\nhostname,group,ip_address,os_distro,os_version,hw_arch,time,changed,unreachable,failed,details\n{# Loop through all hosts #}\n{% for host in ansible_play_hosts_all | sort %}\n{# System time #}\n{% if hostvars[host]['ansible_facts']['date_time'] is not defined %}\n{% set sys_time = '' %}\n{% else %}\n{% set sys_time = hostvars[host]['ansible_facts']['date_time']['date'] + ' ' + hostvars[host]['ansible_facts']['date_time']['time'] %}\n{% endif %}\n{# Package details #}\n{% set ns = namespace( details = [], changed = false ) %}\n{% for pkg_name in client_sw_pkg_state %}\n{% set pkg_file = hostvars[host]['ansible_facts']['sas_client_sw_pkgs']['packages'][pkg_name]['file'] | default('none', true) %}\n{% set pkg_vers = hostvars[host]['ansible_facts']['sas_client_sw_pkgs']['packages'][pkg_name]['vers'] | default('none', true) %}\n{% set pkg_req = hostvars[host]['ansible_facts']['sas_client_sw_' + pkg_name + 'req'] | default('none', true) %}\n{% set pkg_act = hostvars[host]['ansible_facts']['sas_client_sw' + pkg_name + 'act'] | default('none', true) %}\n{% set pkg_vers_beg = hostvars[host]['ansible_facts']['sas_client_sw' + pkg_name + 'vers_beg'] | default('none', true) %}\n{% set pkg_vers_end = hostvars[host]['ansible_facts']['sas_client_sw' + pkg_name + '_vers_end'] | default('none', true) %}\n{% set pkg_detail = {'key': pkg_name, 'value': {'installer_file': pkg_file, 'installer_version': pkg_vers, 'state_request': pkg_req, 'state_change': pkg_act, 'version_begin': pkg_vers_beg, 'version_end': pkg_vers_end}} %}\n{% if pkg_file != 'none' or pkg_act != 'none' %}\n{% set ns.details = ns.details + [pkg_detail] %}\n{% endif %}\n{% if pkg_act not in ['none', 'failed'] %}\n{% set ns.changed = true %}\n{% endif %}\n{% endfor %}\n{% set package_details = ns.details | items2dict %}\n{# Status #}\n{% set changed = ns.changed %}\n{% set unreachable = hostvars[host]['ansible_facts']['sas_client']['unreachable'] | default(False) %}\n{% set failed = hostvars[host]['ansible_facts']['sas_client']['failed'] | default(True) %}\n{% set msg = hostvars[host]['ansible_facts']['sas_client']['msg'] | default('Unexpected error occurred') %}\n{# Details #}\n{% set details = {\n 'hostname': host,\n 'msg': msg,\n 'unreachable': unreachable,\n 'failed': failed,\n 'changed': changed,\n 'packages': package_details\n }\n%}\n{# Format details #}\n{% if details %}\n{% if client_sw_reports_details_format | lower == 'json' %}\n{% set details = details | to_nice_json(indent=2) | replace("\"", "\"\"") %}\n{% else %}\n{% set details = details | to_nice_yaml(indent=2, width=160) | replace("\"", "\"\"") %}\n{% endif %}\n{% else %}\n{% set details = '' %}\n{% endif %}\n{# Print CSV line #}\n{{ '%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,"%s"' | format(\nhost,\nhostvars[host]['group_names'] | join(', '),\nhostvars[host]['ansible_facts']['default_ipv4']['address'] | default(),\nhostvars[host]['ansible_facts']['distribution'] | default(),\nhostvars[host]['ansible_facts']['distribution_version'] | default(),\nhostvars[host]['ansible_facts']['architecture'] | default(),\nsys_time,\nchanged,\nunreachable,\nfailed,\ndetails\n)}}\n{% endfor %}"
}

[cschar7]

Maybe not since ansible still shows the python version as 2.7.5

[user1@server1 ~]$ ansible --version
ansible 2.9.15
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/home/blsadmin/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /bin/ansible
python version = 2.7.5 (default, Aug 13 2020, 02:51:10) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
[user1@server1 ~]$

[llnagy76]

While I was trying to reproduce the error I tried to change the python version ansible uses on the controller node because my ansible uses python 3 and I would have liked it to use python 2.7.5 instead. Then I found this in ansible doc:
Individual Linux distribution packages may be packaged for Python2 or Python3. When running from distro packages you’ll only be able to use Ansible with the Python version for which it was installed. Sometimes distros will provide a means of installing for several Python versions (via a separate package or via some commands that are run after install). You’ll need to check with your distro to see if that applies in your case.
I could not change the python version ansible uses on the controller node and it sounds like it is not possible. Or at least not possible by simply installing another python version and modifying some configuration settings.