Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

netconf connections do not support huge_tree #255

Open
mweinelt opened this issue Apr 8, 2021 · 2 comments · May be fixed by #685
Open

netconf connections do not support huge_tree #255

mweinelt opened this issue Apr 8, 2021 · 2 comments · May be fixed by #685
Assignees
@rohitthakur2590

Comments

@mweinelt
Copy link

mweinelt commented Apr 8, 2021

SUMMARY

This issue is similar to ansible/ansible#69512, but the issue is not in junos_rpc or junos_command, and as such relevant to this repository since it relies on a change to the netconf connection.

The ncclient netconf connection needs to be opened with huge_tree=True, which was added to the library in 2019 (ncclient/ncclient#319) and requires ncclient>=0.6.5.

This is required to support fetching large xml trees from network devices. This issue happens e.g. when fetching support information from a JunOS router.

ISSUE TYPE
  • Bug Report
COMPONENT NAME

netconf

ANSIBLE VERSION
ansible 2.9.19
  config file = /home/hexa/git/man-da/ansible-network/ansible.cfg
  configured module search path = ['/home/hexa/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible
  executable location = /home/hexa/git/man-da/ansible-network/.env/bin/ansible
  python version = 3.8.8 (default, Feb 19 2021, 11:04:50) [GCC 10.2.0]
CONFIGURATION
DEFAULT_HOST_LIST(/home/hexa/git/man-da/ansible-network/ansible.cfg) = ['/home/hexa/git/man-da/ansible-network/inventory']
DEFAULT_STDOUT_CALLBACK(/home/hexa/git/man-da/ansible-network/ansible.cfg) = yaml
PERSISTENT_COMMAND_TIMEOUT(/home/hexa/git/man-da/ansible-network/ansible.cfg) = 600
OS / ENVIRONMENT

NixOS Unstable
Tested against Junos MX240

# pip freeze
ansible==2.9.19
appdirs @ file:///build/appdirs-1.4.4/dist/appdirs-1.4.4-py2.py3-none-any.whl
bcrypt==3.2.0
cffi==1.14.5
cryptography==3.4.7
distlib @ file:///build/distlib-0.3.1/dist/distlib-0.3.1-py3-none-any.whl
filelock @ file:///build/filelock-3.0.12/dist/filelock-3.0.12-py3-none-any.whl
Jinja2==2.11.3
lxml==4.6.3
MarkupSafe==1.1.1
ncclient==0.6.10
paramiko==2.7.2
pycparser==2.20
PyNaCl==1.4.0
PyYAML==5.4.1
six @ file:///build/six-1.15.0/dist/six-1.15.0-py2.py3-none-any.whl
virtualenv @ file:///build/virtualenv-20.2.2/dist/virtualenv-20.2.2-py2.py3-none-any.whl
STEPS TO REPRODUCE
  1. Run the rpc call get-support-information against a JunOS router (Mx240 in our case)
    - name: "Get support information via RPC call"
      junos_rpc:
        rpc: get-support-information
        output: text
      register: rsi
EXPECTED RESULTS

To get the support information in the rsi variable.

ACTUAL RESULTS
task path: /home/hexa/git/man-da/ansible-network/get_support.yml:30
<core1.fra1.ix.f.man-da.net> attempting to start connection
<core1.fra1.ix.f.man-da.net> using connection plugin netconf
<core1.fra1.ix.f.man-da.net> found existing local domain socket, using it!
<core1.fra1.ix.f.man-da.net> 
<core1.fra1.ix.f.man-da.net> local domain socket path is /home/hexa/.ansible/pc/a6233c2946
<core1.fra1.ix.f.man-da.net> Using network group action junos for junos_rpc
<core1.fra1.ix.f.man-da.net> ESTABLISH LOCAL CONNECTION FOR USER: hexa
<core1.fra1.ix.f.man-da.net> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi `"&& mkdir "` echo /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225 `" && echo ansible-tmp-1617890385.6309385-12393-213464275606225="` echo /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225 `" ) && sleep 0'
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/network/junos/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/network/common/netconf.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/network/common/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/network/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/basic.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/six/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/network/junos/junos.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/connection.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/_text.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/json.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/_collections_compat.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/collections.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/text/formatters.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/parsing/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/parsing/convert_bool.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/compat/selectors.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/_json_compat.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/pycompat24.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/validation.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/text/converters.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/process.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/text/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/compat/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/sys_info.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/parameters.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/_utils.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/common/file.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/compat/_selectors2.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/distro/__init__.py
Using module_utils file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/module_utils/distro/_distro.py
Using module file /home/hexa/git/man-da/ansible-network/.env/lib/python3.8/site-packages/ansible/modules/network/junos/junos_rpc.py
<core1.fra1.ix.f.man-da.net> PUT /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/tmpzos1lj_7 TO /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py
<core1.fra1.ix.f.man-da.net> EXEC /bin/sh -c 'chmod u+x /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/ /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py && sleep 0'
<core1.fra1.ix.f.man-da.net> EXEC /bin/sh -c '/usr/bin/env python3 /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py && sleep 0'
<core1.fra1.ix.f.man-da.net> EXEC /bin/sh -c 'rm -f -r /home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 83, in parse_rpc_error
  File "src/lxml/etree.pyx", line 3237, in lxml.etree.fromstring
  File "src/lxml/parser.pxi", line 1896, in lxml.etree._parseMemoryDocument
  File "src/lxml/parser.pxi", line 1784, in lxml.etree._parseDoc
  File "src/lxml/parser.pxi", line 1141, in lxml.etree._BaseParser._parseDoc
  File "src/lxml/parser.pxi", line 615, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
  File "<string>", line 1
lxml.etree.XMLSyntaxError: Start tag expected, '<' not found, line 1, column 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py", line 102, in <module>
    _ansiballz_main()
  File "/home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible.modules.network.junos.junos_rpc', init_globals=None, run_name='__main__', alter_sys=True)
  File "/nix/store/5kznrncnp9v828a7i723j72bh9wkbkg7-python3-3.8.8/lib/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/nix/store/5kznrncnp9v828a7i723j72bh9wkbkg7-python3-3.8.8/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/nix/store/5kznrncnp9v828a7i723j72bh9wkbkg7-python3-3.8.8/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/modules/network/junos/junos_rpc.py", line 173, in <module>
  File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/modules/network/junos/junos_rpc.py", line 154, in main
  File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 53, in exec_rpc
  File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 76, in __rpc__
  File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 108, in parse_rpc_error
ansible.module_utils.connection.ConnectionError: b'xmlSAX2Characters: huge text node, line 127359, column 84 (<string>, line 127359)'
fatal: [core1.fra1.ix.f.man-da.net]: FAILED! => changed=false 
  module_stderr: |-
    Traceback (most recent call last):
      File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 83, in parse_rpc_error
      File "src/lxml/etree.pyx", line 3237, in lxml.etree.fromstring
      File "src/lxml/parser.pxi", line 1896, in lxml.etree._parseMemoryDocument
      File "src/lxml/parser.pxi", line 1784, in lxml.etree._parseDoc
      File "src/lxml/parser.pxi", line 1141, in lxml.etree._BaseParser._parseDoc
      File "src/lxml/parser.pxi", line 615, in lxml.etree._ParserContext._handleParseResultDoc
      File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
      File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
      File "<string>", line 1
    lxml.etree.XMLSyntaxError: Start tag expected, '<' not found, line 1, column 1
  
    During handling of the above exception, another exception occurred:
  
    Traceback (most recent call last):
      File "/home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py", line 102, in <module>
        _ansiballz_main()
      File "/home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py", line 94, in _ansiballz_main
        invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
      File "/home/hexa/.ansible/tmp/ansible-local-12266v3avvaqi/ansible-tmp-1617890385.6309385-12393-213464275606225/AnsiballZ_junos_rpc.py", line 40, in invoke_module
        runpy.run_module(mod_name='ansible.modules.network.junos.junos_rpc', init_globals=None, run_name='__main__', alter_sys=True)
      File "/nix/store/5kznrncnp9v828a7i723j72bh9wkbkg7-python3-3.8.8/lib/python3.8/runpy.py", line 207, in run_module
        return _run_module_code(code, init_globals, run_name, mod_spec)
      File "/nix/store/5kznrncnp9v828a7i723j72bh9wkbkg7-python3-3.8.8/lib/python3.8/runpy.py", line 97, in _run_module_code
        _run_code(code, mod_globals, init_globals,
      File "/nix/store/5kznrncnp9v828a7i723j72bh9wkbkg7-python3-3.8.8/lib/python3.8/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/modules/network/junos/junos_rpc.py", line 173, in <module>
      File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/modules/network/junos/junos_rpc.py", line 154, in main
      File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 53, in exec_rpc
      File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 76, in __rpc__
      File "/run/user/1000/ansible_junos_rpc_payload_lpas__tq/ansible_junos_rpc_payload.zip/ansible/module_utils/network/common/netconf.py", line 108, in parse_rpc_error
    ansible.module_utils.connection.ConnectionError: b'xmlSAX2Characters: huge text node, line 127359, column 84 (<string>, line 127359)'
  module_stdout: ''
  msg: |-
    MODULE FAILURE
    See stdout/stderr for the exact error
  rc: 1
...ignoring
@irregulator
Copy link

irregulator commented Oct 4, 2021 •
edited
Loading

Hello there,

we've bumped in the very same issue while using junos_config module from junipernetworks.junos ansible collection.

Here's the task:

- name: "Just a task"
  junipernetworks.junos.junos_config:
    lines:
      - 'set policy-options policy-statement SOME-POLICY then SOMETHING"'

and here's the traceback:

TASK [Just a task] *******************************************************************************************************************
task path: /home/alexandros/src/ansible-playbooks/network/playbooks/_junos_something.yml:4
<juniper-mx240-router.exoscale.net> attempting to start connection
<juniper-mx240-router.exoscale.net> using connection plugin netconf
<juniper-mx240-router.exoscale.net> found existing local domain socket, using it!
<juniper-mx240-router.exoscale.net> 
<juniper-mx240-router.exoscale.net> local domain socket path is /home/alexandros/.ansible/pc/6d4fe786ab
<juniper-mx240-router.exoscale.net> ESTABLISH LOCAL CONNECTION FOR USER: alexandros
<juniper-mx240-router.exoscale.net> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/alexandros/.ansible/tmp/ansible-local-1036169bwk3lya/ansible-tmp-1633087881.8289015-7373941123697 `" && echo ansible-tmp-1633087881.8289015-7373941123697="` echo /home/alexandros/.ansible/tmp/ansible-local-1036169bwk3lya/ansible-tmp-1633087881.8289015-7373941123697 `" ) && sleep 0'
Using module_utils file ansible_collections/ansible/netcommon/plugins/module_utils/network/common/netconf
Using module_utils file ansible_collections/junipernetworks/junos/plugins/module_utils/network/junos/junos
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/six/__init__.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/_text.py
Using module_utils file ansible_collections/ansible
Using module_utils file ansible_collections/ansible/netcommon
Using module_utils file ansible_collections/junipernetworks
Using module_utils file ansible_collections/junipernetworks/junos/plugins/module_utils/network
Using module_utils file ansible_collections/junipernetworks/junos
Using module_utils file ansible_collections/junipernetworks/junos/plugins
Using module_utils file ansible_collections
Using module_utils file ansible_collections/ansible/netcommon/plugins
Using module_utils file ansible_collections/junipernetworks/junos/plugins/module_utils/network/junos
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/basic.py
Using module_utils file ansible_collections/ansible/netcommon/plugins/module_utils
Using module_utils file ansible_collections/junipernetworks/junos/plugins/module_utils
Using module_utils file ansible_collections/ansible/netcommon/plugins/module_utils/network
Using module_utils file ansible_collections/ansible/netcommon/plugins/module_utils/network/common
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/connection.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/__init__.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/json.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/collections.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/_collections_compat.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/text/formatters.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/text/converters.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/_utils.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/parsing/__init__.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/sys_info.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/file.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/_json_compat.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/text/__init__.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/parsing/convert_bool.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/validation.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/parameters.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/pycompat24.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/common/process.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/distro/__init__.py
Using module_utils file /home/alexandros/src/ansible-playbooks/network/.venv/lib/python3.8/site-packages/ansible/module_utils/distro/_distro.py
Using module file /home/alexandros/.ansible/collections/ansible_collections/junipernetworks/junos/plugins/modules/junos_config.py
<juniper-mx240-router.exoscale.net> PUT /home/alexandros/.ansible/tmp/ansible-local-1036169bwk3lya/tmp4e81lxgi TO /home/alexandros/.ansible/tmp/ansible-local-1036169bwk3lya/ansible-tmp-1633087881.8289015-7373941123697/AnsiballZ_junos_config.py
<juniper-mx240-router.exoscale.net> EXEC /bin/sh -c 'chmod u+x /home/alexandros/.ansible/tmp/ansible-local-1036169bwk3lya/ansible-tmp-1633087881.8289015-7373941123697/ /home/alexandros/.ansible/tmp/ansible-local-1036169bwk3lya/ansible-tmp-1633087881.8289015-7373941123697/AnsiballZ_junos_config.py && sleep 0'
<juniper-mx240-router.exoscale.net> EXEC /bin/sh -c '/usr/bin/python2.7 /home/alexandros/.ansible/tmp/ansible-local-1036169bwk3lya/ansible-tmp-1633087881.8289015-7373941123697/AnsiballZ_junos_config.py && sleep 0'
<juniper-mx240-router.exoscale.net> EXEC /bin/sh -c 'rm -f -r /home/alexandros/.ansible/tmp/ansible-local-1036169bwk3lya/ansible-tmp-1633087881.8289015-7373941123697/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/home/alexandros/.ansible/tmp/ansible-local-1036169bwk3lya/ansible-tmp-1633087881.8289015-7373941123697/AnsiballZ_junos_config.py", line 102, in <module>
    _ansiballz_main()
  File "/home/alexandros/.ansible/tmp/ansible-local-1036169bwk3lya/ansible-tmp-1633087881.8289015-7373941123697/AnsiballZ_junos_config.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/alexandros/.ansible/tmp/ansible-local-1036169bwk3lya/ansible-tmp-1633087881.8289015-7373941123697/AnsiballZ_junos_config.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible_collections.junipernetworks.junos.plugins.modules.junos_config', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/lib/python2.7/runpy.py", line 188, in run_module
    fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 82, in _run_module_code
    mod_name, mod_fname, mod_loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/tmp/ansible_junipernetworks.junos.junos_config_payload_Ckd0U0/ansible_junipernetworks.junos.junos_config_payload.zip/ansible_collections/junipernetworks/junos/plugins/modules/junos_config.py", line 534, in <module>
  File "/tmp/ansible_junipernetworks.junos.junos_config_payload_Ckd0U0/ansible_junipernetworks.junos.junos_config_payload.zip/ansible_collections/junipernetworks/junos/plugins/modules/junos_config.py", line 493, in main
  File "/tmp/ansible_junipernetworks.junos.junos_config_payload_Ckd0U0/ansible_junipernetworks.junos.junos_config_payload.zip/ansible_collections/junipernetworks/junos/plugins/modules/junos_config.py", line 392, in configure_device
  File "/tmp/ansible_junipernetworks.junos.junos_config_payload_Ckd0U0/ansible_junipernetworks.junos.junos_config_payload.zip/ansible_collections/junipernetworks/junos/plugins/modules/junos_config.py", line 353, in filter_delete_statements
  File "/tmp/ansible_junipernetworks.junos.junos_config_payload_Ckd0U0/ansible_junipernetworks.junos.junos_config_payload.zip/ansible_collections/junipernetworks/junos/plugins/module_utils/network/junos/junos.py", line 225, in get_configuration
  File "/tmp/ansible_junipernetworks.junos.junos_config_payload_Ckd0U0/ansible_junipernetworks.junos.junos_config_payload.zip/ansible_collections/ansible/netcommon/plugins/module_utils/network/common/netconf.py", line 85, in __rpc__
  File "src/lxml/etree.pyx", line 3235, in lxml.etree.fromstring
  File "src/lxml/parser.pxi", line 1876, in lxml.etree._parseMemoryDocument
  File "src/lxml/parser.pxi", line 1764, in lxml.etree._parseDoc
  File "src/lxml/parser.pxi", line 1127, in lxml.etree._BaseParser._parseDoc
  File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 640, in lxml.etree._raiseParseError
  File "<string>", line 161541
lxml.etree.XMLSyntaxError: xmlSAX2Characters: huge text node, line 161541, column 1

As can be deduced from the traceback, junipernetworks.junos.junos_config actually employs the __rpc__ method of NetconfConnection class, as found in ansible.netcommon collection.

I've been working on a PoC fix for this issue and willing to open a PR soon. Spoiler: main...irregulator:netconf/support_XML_huge_tree

As soon as XML huge_tree support is available in ansible.netcommon we can later leverage it in junipernetworks.junos modules. I'm really open to feedback and ideas.

@irregulator irregulator mentioned this issue Oct 4, 2021
Closed
@laxxie
Copy link

laxxie commented Nov 12, 2024

Hejhej,
is there any update? We are running into the same problems with the junos_config module because of large filter-lists

mweinelt added a commit to mweinelt/ansible.netcommon that referenced this issue Feb 4, 2025
@mweinelt
Add huge_tree support for RPC calls
e367361 
When retrieving large XML responses over Netconf we need to use lxml
with huge_trees, because the default XML parser bails out.

Fixes: ansible-collections#255
@mweinelt mweinelt linked a pull request Feb 4, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rohitthakur2590 rohitthakur2590

Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@mweinelt @irregulator @rohitthakur2590 @laxxie