Template lookup start string breaks environment vars parsing

[ericbaum]

SUMMARY

When we create a playbook with environment variables defined from input variables, if one of the tasks in this playbooks use a template lookup plugin with a start/end string different from default, it also changes the start/end string used to parse the environment variable and breaks its parsing.

ISSUE TYPE

• Bug Report

COMPONENT NAME

Template Lookup Plugin

ANSIBLE VERSION

ansible 2.8.3
  config file = None
  configured module search path = [u'/home/USER/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.16 (default, Apr  6 2019, 01:42:57) [GCC 8.3.0]

CONFIGURATION

OS / ENVIRONMENT

Distributor ID: Ubuntu
Description: Ubuntu 19.04
Release: 19.04
Codename: disco

STEPS TO REPRODUCE

Create the playbook presented below, also create an empty file called dummy.yaml.j2.

Execute the playbook with ansible-playbook playbook.yaml -e test_entry=lalala

# Bug example
- hosts: localhost
  gather_facts: false
  
  environment:
    TEST_ENV: "{{ test_entry }}"

  tasks:

  - name: Test 1
    command: "echo ${TEST_ENV} {{ lookup('template', 'dummy.yaml.j2') }}"
    register: output
  
  - debug:
      msg: "{{ output.stdout }}"

  - name: Test 2
    command: "echo ${TEST_ENV} {{ lookup('template', 'dummy.yaml.j2', variable_start_string='[%', variable_end_string='%]') }}"
    register: output

  - debug:
      msg: "{{ output.stdout }}"

  - name: Test 3
    command: "echo ${TEST_ENV} {{ lookup('template', 'dummy.yaml.j2', variable_start_string='[%', variable_end_string='%]') }}"
    environment:
      TEST_ENV: "[% test_entry %]"
    register: output

  - debug:
      msg: "{{ output.stdout }}"

EXPECTED RESULTS

The expected result was that Debug 1 and 2 output the correct result "lalala" and Debug 3
output output "[% test_entry %]" as it is just a workaround to the bug.

ACTUAL RESULTS

What happened was that debug 1 had the right output, debug 2 had the variable parsing broken by the lookup and debug 3 worked as a workaround to the issue, which should not be necessary.

USER@USER-note:~/workspace/tf-infra$ansible-playbook -e test_entry=lalala playbooks/test.yaml -vvvv
ansible-playbook 2.8.3
  config file = None
  configured module search path = [u'/home/USER/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible-playbook
  python version = 2.7.16 (default, Apr  6 2019, 01:42:57) [GCC 8.3.0]
No config file found; using defaults
setting up inventory plugins
host_list declined parsing /etc/ansible/hosts as it did not pass it's verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
script declined parsing /etc/ansible/hosts as it did not pass it's verify_file() method
auto declined parsing /etc/ansible/hosts as it did not pass it's verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
yaml declined parsing /etc/ansible/hosts as it did not pass it's verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
ini declined parsing /etc/ansible/hosts as it did not pass it's verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
toml declined parsing /etc/ansible/hosts as it did not pass it's verify_file() method
 [WARNING]: No inventory was parsed, only implicit localhost is available

 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

Loading callback plugin default of type stdout, v2.0 from /usr/local/lib/python2.7/dist-packages/ansible/plugins/callback/default.pyc

PLAYBOOK: test.yaml ***************************************************************************************************************************************************************
Positional arguments: playbooks/test.yaml
become_method: sudo
inventory: (u'/etc/ansible/hosts',)
forks: 5
tags: (u'all',)
extra_vars: (u'test_entry=lalala',)
verbosity: 4
connection: smart
timeout: 10
1 plays in playbooks/test.yaml

PLAY [localhost] ******************************************************************************************************************************************************************
META: ran handlers

TASK [Test 1] *********************************************************************************************************************************************************************
task path: /home/USER/workspace/playbooks/test.yaml:10
File lookup using /home/USER/workspace/playbooks/dummy.yaml.j2 as file
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: USER
<127.0.0.1> EXEC /bin/sh -c 'echo ~USER && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/USER/.ansible/tmp/ansible-tmp-1565035196.17-187587540004383 `" && echo ansible-tmp-1565035196.17-187587540004383="` echo /home/USER/.ansible/tmp/ansible-tmp-1565035196.17-187587540004383 `" ) && sleep 0'
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/commands/command.py
<127.0.0.1> PUT /home/USER/.ansible/tmp/ansible-local-299637oQZ14/tmpFL3AO6 TO /home/USER/.ansible/tmp/ansible-tmp-1565035196.17-187587540004383/AnsiballZ_command.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/USER/.ansible/tmp/ansible-tmp-1565035196.17-187587540004383/ /home/USER/.ansible/tmp/ansible-tmp-1565035196.17-187587540004383/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'TEST_ENV=lalala /usr/bin/python /home/USER/.ansible/tmp/ansible-tmp-1565035196.17-187587540004383/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/USER/.ansible/tmp/ansible-tmp-1565035196.17-187587540004383/ > /dev/null 2>&1 && sleep 0'
changed: [localhost] => {
    "changed": true, 
    "cmd": [
        "echo", 
        "${TEST_ENV}"
    ], 
    "delta": "0:00:00.002597", 
    "end": "2019-08-05 16:59:56.387369", 
    "invocation": {
        "module_args": {
            "_raw_params": "echo ${TEST_ENV} ", 
            "_uses_shell": false, 
            "argv": null, 
            "chdir": null, 
            "creates": null, 
            "executable": null, 
            "removes": null, 
            "stdin": null, 
            "stdin_add_newline": true, 
            "strip_empty_ends": true, 
            "warn": true
        }
    }, 
    "rc": 0, 
    "start": "2019-08-05 16:59:56.384772", 
    "stderr": "", 
    "stderr_lines": [], 
    "stdout": "lalala", 
    "stdout_lines": [
        "lalala"
    ]
}

TASK [debug] **********************************************************************************************************************************************************************
task path: /home/USER/workspace/playbooks/test.yaml:14
ok: [localhost] => {
    "msg": "lalala"
}

TASK [Test 2] *********************************************************************************************************************************************************************
task path: /home/USER/workspace/playbooks/test.yaml:17
File lookup using /home/USER/workspace/playbooks/dummy.yaml.j2 as file
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: USER
<127.0.0.1> EXEC /bin/sh -c 'echo ~USER && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/USER/.ansible/tmp/ansible-tmp-1565035196.46-279367437516145 `" && echo ansible-tmp-1565035196.46-279367437516145="` echo /home/USER/.ansible/tmp/ansible-tmp-1565035196.46-279367437516145 `" ) && sleep 0'
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/commands/command.py
<127.0.0.1> PUT /home/USER/.ansible/tmp/ansible-local-299637oQZ14/tmpriIGjL TO /home/USER/.ansible/tmp/ansible-tmp-1565035196.46-279367437516145/AnsiballZ_command.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/USER/.ansible/tmp/ansible-tmp-1565035196.46-279367437516145/ /home/USER/.ansible/tmp/ansible-tmp-1565035196.46-279367437516145/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'TEST_ENV='"'"'{{ test_entry }}'"'"' /usr/bin/python /home/USER/.ansible/tmp/ansible-tmp-1565035196.46-279367437516145/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/USER/.ansible/tmp/ansible-tmp-1565035196.46-279367437516145/ > /dev/null 2>&1 && sleep 0'
changed: [localhost] => {
    "changed": true, 
    "cmd": [
        "echo", 
        "${TEST_ENV}"
    ], 
    "delta": "0:00:00.002174", 
    "end": "2019-08-05 16:59:56.551546", 
    "invocation": {
        "module_args": {
            "_raw_params": "echo ${TEST_ENV} ", 
            "_uses_shell": false, 
            "argv": null, 
            "chdir": null, 
            "creates": null, 
            "executable": null, 
            "removes": null, 
            "stdin": null, 
            "stdin_add_newline": true, 
            "strip_empty_ends": true, 
            "warn": true
        }
    }, 
    "rc": 0, 
    "start": "2019-08-05 16:59:56.549372", 
    "stderr": "", 
    "stderr_lines": [], 
    "stdout": "{{ test_entry }}", 
    "stdout_lines": [
        "{{ test_entry }}"
    ]
}

TASK [debug] **********************************************************************************************************************************************************************
task path: /home/USER/workspace/playbooks/test.yaml:21
ok: [localhost] => {
    "msg": "{{ test_entry }}"
}

TASK [Test 3] *********************************************************************************************************************************************************************
task path: /home/USER/workspace/playbooks/test.yaml:24
File lookup using /home/USER/workspace/playbooks/dummy.yaml.j2 as file
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: USER
<127.0.0.1> EXEC /bin/sh -c 'echo ~USER && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/USER/.ansible/tmp/ansible-tmp-1565035196.61-117891973732187 `" && echo ansible-tmp-1565035196.61-117891973732187="` echo /home/USER/.ansible/tmp/ansible-tmp-1565035196.61-117891973732187 `" ) && sleep 0'
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/commands/command.py
<127.0.0.1> PUT /home/USER/.ansible/tmp/ansible-local-299637oQZ14/tmpKaHwr_ TO /home/USER/.ansible/tmp/ansible-tmp-1565035196.61-117891973732187/AnsiballZ_command.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/USER/.ansible/tmp/ansible-tmp-1565035196.61-117891973732187/ /home/USER/.ansible/tmp/ansible-tmp-1565035196.61-117891973732187/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'TEST_ENV=lalala /usr/bin/python /home/USER/.ansible/tmp/ansible-tmp-1565035196.61-117891973732187/AnsiballZ_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/USER/.ansible/tmp/ansible-tmp-1565035196.61-117891973732187/ > /dev/null 2>&1 && sleep 0'
changed: [localhost] => {
    "changed": true, 
    "cmd": [
        "echo", 
        "${TEST_ENV}"
    ], 
    "delta": "0:00:00.002216", 
    "end": "2019-08-05 16:59:56.713032", 
    "invocation": {
        "module_args": {
            "_raw_params": "echo ${TEST_ENV} ", 
            "_uses_shell": false, 
            "argv": null, 
            "chdir": null, 
            "creates": null, 
            "executable": null, 
            "removes": null, 
            "stdin": null, 
            "stdin_add_newline": true, 
            "strip_empty_ends": true, 
            "warn": true
        }
    }, 
    "rc": 0, 
    "start": "2019-08-05 16:59:56.710816", 
    "stderr": "", 
    "stderr_lines": [], 
    "stdout": "lalala", 
    "stdout_lines": [
        "lalala"
    ]
}

TASK [debug] **********************************************************************************************************************************************************************
task path: /home/USER/workspace/playbooks/test.yaml:30
ok: [localhost] => {
    "msg": "lalala"
}
META: ran handlers
META: ran handlers

PLAY RECAP ************************************************************************************************************************************************************************
localhost                  : ok=6    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

[ansibot]

Files identified in the description:

• lib/ansible/plugins/lookup/template.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help