network: netconf: support XML huge_tree for RPC calls

[irregulator]

SUMMARY

__rpc__ method of NetconfConnection class employs fromstring function to parse the XML response. If a node in the XML tree is too big, the parser fails with:

lxml.etree.XMLSyntaxError: xmlSAX2Characters: huge text node

This is actually a libxml2 security feature that can be optionally tweaked with the huge_tree option of the XML parser.

Module junos_config of the junipernetworks.junos collection employs NetconfConnection.__rpc__ to get the configuration from a device. There can be a case when configuration size will exceed the XML parser limit and the module will bail out.

To facilitate such cases it should be possible for the RPC method caller to enable the huge_tree option, which defaults to false. Let the method support that by passing an appropriate parser to fromstring function.

Fixes #255

ISSUE TYPE

• Bugfix Pull Request

COMPONENT NAME

netconf RPC

ADDITIONAL INFORMATION

libxml huge_tree option allows to overcome XML parser errors like:

lxml.etree.XMLSyntaxError: xmlSAX2Characters: huge text node

NetconfConnection.__rpc__ method is called by junipernetworks.junos.junos_config module. So this issue can be spotted when managing Juniper network devices with largish configuration.

[irregulator]

While this approach enables huge_tree only for NetconfConnection RPC calls, someone could argue that we could also enable this for any netconf connection. I opted for the currently suggested change as being simpler and less intrusive, allowing each RPC caller to optionally enable XML huge_tree.

NetconfConnection has not unit tests to extend, as far as I can tell.

I'd welcome feedback or different approaches.

[mweinelt]

Can someone review this change, please? The ci logs are 404ing for me right now, so it should probably be retriggered by rebasing.

[softwarefactory-project-zuul[bot]]

Build failed. https://ansible.softwarefactory-project.io/zuul/buildset/5944f952218b4f23b7ba971ff6d5f3db

:x: ansible-test-network-integration-junos-vsrx-netconf-python36-stable29 FAILURE in 13m 59s (non-voting) :heavy_check_mark: ansible-test-network-integration-junos-vsrx-netconf-python36-stable211 SUCCESS in 1h 26m 43s :heavy_check_mark: ansible-test-network-integration-junos-vsrx-netconf-python39-stable212 SUCCESS in 1h 07m 55s :x: ansible-test-network-integration-junos-vsrx-netconf-python39 RETRY_LIMIT in 5m 56s :x: ansible-test-network-integration-junos-vsrx-network_cli-python36-stable29 FAILURE in 25m 22s (non-voting) :heavy_check_mark: ansible-test-network-integration-junos-vsrx-network_cli-python36-stable211 SUCCESS in 19m 03s :heavy_check_mark: ansible-test-network-integration-junos-vsrx-network_cli-python39-stable212 SUCCESS in 20m 28s :x: ansible-test-network-integration-junos-vsrx-network_cli-python39 RETRY_LIMIT in 12m 05s :x: ansible-test-network-integration-junos-vsrx-network_cli-libssh-python36-stable29 FAILURE in 18m 16s (non-voting) :heavy_check_mark: ansible-test-network-integration-junos-vsrx-network_cli-libssh-python36-stable211 SUCCESS in 19m 29s :heavy_check_mark: ansible-test-network-integration-junos-vsrx-network_cli-libssh-python39-stable212 SUCCESS in 16m 17s :x: ansible-test-network-integration-junos-vsrx-network_cli-libssh-python39 RETRY_LIMIT in 10m 25s :x: ansible-test-network-integration-vyos-paramiko-python39-devel NODE_FAILURE Node request 200-0007214583 failed in 0s :x: ansible-test-network-integration-vyos-paramiko-python39-stable214 NODE_FAILURE Node request 200-0007214584 failed in 0s :x: ansible-test-network-integration-vyos-paramiko-python39-stable213 NODE_FAILURE Node request 200-0007214585 failed in 0s :x: ansible-test-network-integration-vyos-paramiko-python39-stable212 NODE_FAILURE Node request 200-0007214586 failed in 0s :x: ansible-test-network-integration-vyos-paramiko-python36-stable29 NODE_FAILURE Node request 200-0007214587 failed in 0s (non-voting) :x: ansible-test-network-integration-vyos-libssh-python39-devel NODE_FAILURE Node request 200-0007214588 failed in 0s :x: ansible-test-network-integration-vyos-libssh-python39-stable214 NODE_FAILURE Node request 200-0007214589 failed in 0s :x: ansible-test-network-integration-vyos-libssh-python39-stable213 NODE_FAILURE Node request 200-0007214590 failed in 0s :x: ansible-test-network-integration-vyos-libssh-python39-stable212 NODE_FAILURE Node request 200-0007214591 failed in 0s :x: ansible-test-network-integration-vyos-libssh-python36-stable29 NODE_FAILURE Node request 200-0007214592 failed in 0s :heavy_check_mark: build-ansible-collection SUCCESS in 13m 29s :x: ansible-test-network-integration-ansible-netcommon-junos-vsrx-netconf-python39 RETRY_LIMIT in 9m 36s :x: ansible-tox-linters FAILURE in 11m 51s :heavy_check_mark: ansible-galaxy-importer SUCCESS in 5m 05s

[softwarefactory-project-zuul[bot]]

Build failed. https://ansible.softwarefactory-project.io/zuul/buildset/18bbc36f5643423fb8f19bb2c77c0ee3

:x: ansible-test-network-integration-junos-vsrx-netconf-python36-stable29 FAILURE in 14m 02s (non-voting) :heavy_check_mark: ansible-test-network-integration-junos-vsrx-netconf-python36-stable211 SUCCESS in 1h 21m 47s :x: ansible-test-network-integration-junos-vsrx-netconf-python39-stable212 RETRY_LIMIT in 3m 03s :x: ansible-test-network-integration-junos-vsrx-netconf-python39 RETRY_LIMIT in 9m 43s :x: ansible-test-network-integration-junos-vsrx-network_cli-python36-stable29 FAILURE in 14m 12s (non-voting) :heavy_check_mark: ansible-test-network-integration-junos-vsrx-network_cli-python36-stable211 SUCCESS in 17m 38s :heavy_check_mark: ansible-test-network-integration-junos-vsrx-network_cli-python39-stable212 SUCCESS in 15m 34s :x: ansible-test-network-integration-junos-vsrx-network_cli-python39 RETRY_LIMIT in 9m 23s :x: ansible-test-network-integration-junos-vsrx-network_cli-libssh-python36-stable29 FAILURE in 17m 45s (non-voting) :heavy_check_mark: ansible-test-network-integration-junos-vsrx-network_cli-libssh-python36-stable211 SUCCESS in 17m 20s :heavy_check_mark: ansible-test-network-integration-junos-vsrx-network_cli-libssh-python39-stable212 SUCCESS in 15m 20s :x: ansible-test-network-integration-junos-vsrx-network_cli-libssh-python39 RETRY_LIMIT in 8m 39s :x: ansible-test-network-integration-vyos-paramiko-python39-devel NODE_FAILURE Node request 200-0007214938 failed in 0s :x: ansible-test-network-integration-vyos-paramiko-python39-stable214 NODE_FAILURE Node request 200-0007214939 failed in 0s :x: ansible-test-network-integration-vyos-paramiko-python39-stable213 NODE_FAILURE Node request 200-0007214940 failed in 0s :x: ansible-test-network-integration-vyos-paramiko-python39-stable212 NODE_FAILURE Node request 200-0007214941 failed in 0s :x: ansible-test-network-integration-vyos-paramiko-python36-stable29 NODE_FAILURE Node request 200-0007214942 failed in 0s (non-voting) :x: ansible-test-network-integration-vyos-libssh-python39-devel NODE_FAILURE Node request 200-0007214943 failed in 0s :x: ansible-test-network-integration-vyos-libssh-python39-stable214 NODE_FAILURE Node request 200-0007214944 failed in 0s :x: ansible-test-network-integration-vyos-libssh-python39-stable213 NODE_FAILURE Node request 200-0007214945 failed in 0s :x: ansible-test-network-integration-vyos-libssh-python39-stable212 NODE_FAILURE Node request 200-0007214946 failed in 0s :x: ansible-test-network-integration-vyos-libssh-python36-stable29 NODE_FAILURE Node request 200-0007214947 failed in 0s :heavy_check_mark: build-ansible-collection SUCCESS in 13m 02s :x: ansible-test-network-integration-ansible-netcommon-junos-vsrx-netconf-python39 RETRY_LIMIT in 3m 39s :x: ansible-tox-linters FAILURE in 11m 33s :heavy_check_mark: ansible-galaxy-importer SUCCESS in 4m 56s