fetch fails on Windows filenames containing dollar sign

[sckfranke]

SUMMARY

fetch module fails on Windows with file names containing dollar signs (winrm FETCH does not quote these from PowerShell). When using |quote or backslash-escaping the dollars, slurp, which is executed by fetch, does handle such paths correctly and therefore fails on pre-quoted ones.

ISSUE TYPE

• Bug Report

COMPONENT NAME

fetch

ANSIBLE VERSION

ansible 2.7.4
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/etc/ansible/plugins/modules', '/etc/venus/common/ansible/modules']
  ansible python module location = /usr/local/ansible/2.7/lib/python3.6/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.6.1 (default, Oct 24 2017, 05:44:23) [GCC 5.3.0]

CONFIGURATION

ANSIBLE_PIPELINING(/etc/ansible/ansible.cfg) = True
CACHE_PLUGIN(/etc/ansible/ansible.cfg) = jsonfile
CACHE_PLUGIN_CONNECTION(/etc/ansible/ansible.cfg) = /var/lib/ansible/facts
CACHE_PLUGIN_TIMEOUT(/etc/ansible/ansible.cfg) = 86400
DEFAULT_ACTION_PLUGIN_PATH(/etc/ansible/ansible.cfg) = ['/etc/ansible/plugins/action', '/etc/venus/common/ansible/action']
DEFAULT_CALLBACK_PLUGIN_PATH(/etc/ansible/ansible.cfg) = ['/etc/ansible/plugins/callback', '/etc/venus/common/ansible/callback']
DEFAULT_CALLBACK_WHITELIST(/etc/ansible/ansible.cfg) = ['log_plays', 'playbook_name', 'json']
DEFAULT_CONNECTION_PLUGIN_PATH(/etc/ansible/ansible.cfg) = ['/etc/ansible/plugins/connection', '/etc/venus/common/ansible/connection']
DEFAULT_FILTER_PLUGIN_PATH(/etc/ansible/ansible.cfg) = ['/etc/ansible/plugins/filter', '/etc/venus/common/ansible/filter']
DEFAULT_FORKS(/etc/ansible/ansible.cfg) = 30
DEFAULT_GATHERING(/etc/ansible/ansible.cfg) = smart
DEFAULT_HOST_LIST(/etc/ansible/ansible.cfg) = ['/etc/ansible/inventory/venus.yml', '/etc/ansible/inventory/hosts.yml']
DEFAULT_INVENTORY_PLUGIN_PATH(/etc/ansible/ansible.cfg) = ['/etc/ansible/plugins/inventory', '/etc/venus/common/ansible/inventory']
DEFAULT_LOG_PATH(/etc/ansible/ansible.cfg) = /var/log/ansible/ansible.log
DEFAULT_LOOKUP_PLUGIN_PATH(/etc/ansible/ansible.cfg) = ['/etc/ansible/plugins/lookup', '/etc/venus/common/ansible/lookup']
DEFAULT_MODULE_PATH(/etc/ansible/ansible.cfg) = ['/etc/ansible/plugins/modules', '/etc/venus/common/ansible/modules']
DEFAULT_POLL_INTERVAL(/etc/ansible/ansible.cfg) = 15
DEFAULT_ROLES_PATH(/etc/ansible/ansible.cfg) = ['/etc/ansible/roles']
DEFAULT_STDOUT_CALLBACK(/etc/ansible/ansible.cfg) = debug
DEFAULT_TRANSPORT(/etc/ansible/ansible.cfg) = venus
DEFAULT_VARS_PLUGIN_PATH(/etc/ansible/ansible.cfg) = ['/etc/ansible/plugins/vars', '/etc/venus/common/ansible/vars']
INVENTORY_ENABLED(/etc/ansible/ansible.cfg) = ['venus', 'yaml', 'advanced_host_list', 'host_list']
RETRY_FILES_ENABLED(/etc/ansible/ansible.cfg) = False

OS / ENVIRONMENT

• Control host: CentOS 7.6
• target: Microsoft Windows Server 2008 R2 Standard Service Pack 1

STEPS TO REPRODUCE

Create a file such as C:\Temp\file_with_$dollar.txt

- name: Get Logs
  hosts: all
  vars:
    log_files:
      - "C:/Temp/testfile_with_$dollar.txt"
  tasks:
  - name: touch test files
    win_file:
       path: "{{ item }}"
       state: touch
    with_items: "{{ log_files }}"
  - name: fetch log files
    fetch:
        src: "{{ item }}"
        dest: "{{ tmp_dir }}/{{ item | basename }}"
        flat: yes
      with_items: "{{ log_files }}"

EXPECTED RESULTS

File gets copied from Windows remote to localhost.

ACTUAL RESULTS

ansible-playbook 2.7.4
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/etc/ansible/plugins/modules', '/etc/venus/common/ansible/modules']
  ansible python module location = /usr/local/ansible/2.7/lib/python3.6/site-packages/ansible
  executable location = /usr/local/bin/ansible-playbook
  python version = 3.6.1 (default, Oct 24 2017, 05:44:23) [GCC 5.3.0]
Using /etc/ansible/ansible.cfg as config file
setting up inventory plugins
Parsed /etc/ansible/inventory/venus.yml inventory source with venus plugin
/etc/ansible/inventory/hosts.yml did not meet venus requirements, check plugin documentation if this is unexpected
Set default localhost to localhost
Parsed /etc/ansible/inventory/hosts.yml inventory source with yaml plugin
Loading callback plugin debug of type stdout, v2.0 from /usr/local/ansible/2.7/lib/python3.6/site-packages/ansible/plugins/callback/debug.py
Loading callback plugin log_plays of type notification, v2.0 from /usr/local/ansible/2.7/lib/python3.6/site-packages/ansible/plugins/callback/log_plays.py

PLAYBOOK: test.yml ***************************************************************************************************************************************************************************
1 plays in test.yml

PLAY [testclient] ****************************************************************************************************************************************************************************
META: ran handlers

TASK [touch test files] **********************************************************************************************************************************************************************
task path: /home/testuser/ansible/test.yml:7
Using module file /usr/local/ansible/2.7/lib/python3.6/site-packages/ansible/modules/windows/win_file.ps1
<testclient> ESTABLISH WINRM CONNECTION FOR USER: testuser@DOMAIN on PORT 5985 TO testclient
checking if winrm_host testclient is an IPv6 address
EXEC (via pipeline wrapper)
changed: [testclient] => (item=C:/Temp/testfile_with_$dollar.txt) => {
    "changed": true,
    "item": "C:/Temp/testfile_with_$dollar.txt"
}

TASK [fetch log files] ***********************************************************************************************************************************************************************
task path: /home/testuser/ansible/packages/co2mo/test.yml:12
Using module file /usr/local/ansible/2.7/lib/python3.6/site-packages/ansible/modules/windows/win_stat.ps1
<testclient> ESTABLISH WINRM CONNECTION FOR USER: testuser@DOMAIN on PORT 5985 TO testclient
checking if winrm_host testclient is an IPv6 address
EXEC (via pipeline wrapper)
<testclient> FETCH "C:\Temp\testfile_with_$dollar.txt" TO "/home/testuser/ansible/testfile_with_$dollar.txt"
Traceback (most recent call last):
  File "/usr/local/ansible/2.7/lib/python3.6/site-packages/ansible/plugins/connection/winrm.py", line 681, in fetch_file
    raise IOError(to_native(result.std_err))
OSError: #< CLIXML
<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04"><S S="Error">The variable '$dollar' cannot be retrieved because it has not been set._x000D__x000A_</S><S S="Error">At line:2 char:32_x000D__x000A_</S><S S="Error">+ $path = "C:\Temp\testfile_with_$dollar.txt"_x000D__x000A_</S><S S="Error">+                                ~~~~~~~_x000D__x000A_</S><S S="Error">    + CategoryInfo          : InvalidOperation: (dollar:String) [], RuntimeExc _x000D__x000A_</S><S S="Error">   eption_x000D__x000A_</S><S S="Error">    + FullyQualifiedErrorId : VariableIsUndefined_x000D__x000A_</S><S S="Error"> _x000D__x000A_</S><S S="Error">The variable '$path' cannot be retrieved because it has not been set._x000D__x000A_</S><S S="Error">At line:3 char:21_x000D__x000A_</S><S S="Error">+ If (Test-Path -Path $path -PathType Leaf)_x000D__x000A_</S><S S="Error">+                     ~~~~~_x000D__x000A_</S><S S="Error">    + CategoryInfo          : InvalidOperation: (path:String) [], RuntimeExcep _x000D__x000A_</S><S S="Error">   tion_x000D__x000A_</S><S S="Error">    + FullyQualifiedErrorId : VariableIsUndefined_x000D__x000A_</S><S S="Error"> _x000D__x000A_</S></Objs>
fatal: [testclient]: FAILED! => {}

MSG:

failed to transfer file to "/home/testuser/ansible//testfile_with_$dollar.txt"


PLAY RECAP ***********************************************************************************************************************************************************************************
testclient                 : ok=1    changed=1    unreachable=0    failed=1   

When replacing src: "{{ item }}" with src: "{{ item|quote }}":

TASK [fetch log files] ***********************************************************************************************************************************************************************
task path: /home/testuser/ansible/test.yml:12
Using module file /usr/local/ansible/2.7/lib/python3.6/site-packages/ansible/modules/windows/win_stat.ps1
<testclient> ESTABLISH WINRM CONNECTION FOR USER: testuser@DOMAIN on PORT 5985 TO testclient
checking if winrm_host testclient is an IPv6 address
EXEC (via pipeline wrapper)
<testclient> FETCH "C:\Temp\testfile_with_$dollar.txt" TO "/home/testuser/ansible/testfile_with_$dollar.txt"
Traceback (most recent call last):
  File "/usr/local/ansible/2.7/lib/python3.6/site-packages/ansible/plugins/connection/winrm.py", line 681, in fetch_file
    raise IOError(to_native(result.std_err))
OSError: #< CLIXML
<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04"><S S="Error">The variable '$dollar' cannot be retrieved because it has not been set._x000D__x000A_</S><S S="Error">At line:2 char:32_x000D__x000A_</S><S S="Error">+ $path = "C:\Temp\testfile_with_$dollar.txt"_x000D__x000A_</S><S S="Error">+                                ~~~~~~~_x000D__x000A_</S><S S="Error">    + CategoryInfo          : InvalidOperation: (dollar:String) [], RuntimeExc _x000D__x000A_</S><S S="Error">   eption_x000D__x000A_</S><S S="Error">    + FullyQualifiedErrorId : VariableIsUndefined_x000D__x000A_</S><S S="Error"> _x000D__x000A_</S><S S="Error">The variable '$path' cannot be retrieved because it has not been set._x000D__x000A_</S><S S="Error">At line:3 char:21_x000D__x000A_</S><S S="Error">+ If (Test-Path -Path $path -PathType Leaf)_x000D__x000A_</S><S S="Error">+                     ~~~~~_x000D__x000A_</S><S S="Error">    + CategoryInfo          : InvalidOperation: (path:String) [], RuntimeExcep _x000D__x000A_</S><S S="Error">   tion_x000D__x000A_</S><S S="Error">    + FullyQualifiedErrorId : VariableIsUndefined_x000D__x000A_</S><S S="Error"> _x000D__x000A_</S></Objs>
fatal: [testclient]: FAILED! => {}

When replacing src: "{{ item }}" with src: "{{ item|replace('$', '\\$') }}":

TASK [fetch log files] ***********************************************************************************************************************************************************************
task path: /home/testuser/ansible/test.yml:12
Using module file /usr/local/ansible/2.7/lib/python3.6/site-packages/ansible/modules/windows/win_stat.ps1
<testclient> ESTABLISH WINRM CONNECTION FOR USER: testuser@DOMAIN on PORT 5985 TO testclient
checking if winrm_host testclient is an IPv6 address
EXEC (via pipeline wrapper)
Using module file /usr/local/ansible/2.7/lib/python3.6/site-packages/ansible/modules/windows/slurp.ps1
EXEC (via pipeline wrapper)
failed: [testclient] (item=C:/Temp/testfile_with_$dollar.txt) => {
    "changed": false,
    "item": "C:/Temp/testfile_with_$dollar.txt"
}

MSG:

Path C:\Temp\testfile_with_\$dollar.txt is not found

WORKAROUND

    - name: create temporary copy of log files
      win_copy:
        remote_src: true
        src: "{{ item }}"
        dest: "{{ item | replace('$', 'DOLLAR') }}.transfer"
        force: no
      with_items: "{{ log_files }}"
    - name: fetch log files
      fetch:
        src: "{{ item | replace('$', 'DOLLAR') }}.transfer"
        dest: "{{ tmp_dir }}/{{ item | basename }}"
        flat: yes
      with_items: "{{ log_files }}"
    - name: remove temporary copy of log files
      win_file:
        path: "{{ item | replace('$', 'DOLLAR') }}.transfer"
        state: absent
      with_items: "{{ log_files }}"

[ansibot]

Files identified in the description:

• lib/ansible/modules/files/fetch.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibot]

Files identified in the description:

• lib/ansible/modules/fetch.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help