ansible-junos-stdlib
ansible-junos-stdlib copied to clipboard
Published
20 hours ago •
Juniper
Juniper
juniper_junos_command - lxml.etree.XMLSyntaxError: StartTag: invalid element name
Issue Type
- Bug Report Module Name: juniper_junos_command
ansible==2.7.7
apache-libcloud==2.4.0
asn1crypto==0.24.0
bcrypt==3.1.6
certifi==2018.8.24
chardet==3.0.4
cryptography==2.6.1
entrypoints==0.3
future==0.18.2
httplib2==0.11.3
idna==2.6
Jinja2==2.10
jmespath==0.9.4
junos-eznc==2.5.3
jxmlease==1.0.3
keyring==17.1.1
keyrings.alt==3.1.1
lockfile==0.12.2
lxml==4.5.2
MarkupSafe==1.1.0
ncclient==0.6.9
netaddr==0.7.19
ntc-templates==1.5.0
ntlm-auth==1.1.0
paramiko==2.4.2
pyasn1==0.4.2
pycrypto==2.6.1
pycurl==7.43.0.2
PyGObject==3.30.4
pykerberos==1.1.14
PyNaCl==1.3.0
pyOpenSSL==19.0.0
pyparsing==2.4.7
pyserial==3.4
PySimpleSOAP==1.16.2
python-apt==1.8.4.1
python-debian==0.1.35
python-debianbts==2.8.2
pywinrm==0.3.0
pyxdg==0.25
PyYAML==5.3.1
reportbug===7.5.3-deb10u1
requests==2.21.0
requests-kerberos==0.11.0
requests-ntlm==1.1.0
scp==0.13.2
SecretStorage==2.3.1
simplejson==3.16.0
six==1.12.0
textfsm==1.1.0
transitions==0.8.2
urllib3==1.24.1
xmltodict==0.11.0
yamlordereddictloader==0.4.0
OS / Environment
SRX110-H2-VA 12.3X48-D105.4
Summary
Using following playbook, an error occurs
- name: Get junos commit diff
hosts: diff-test
connection: local
gather_facts: no
roles:
- Juniper.junos
tasks:
- name: Diff the config
juniper_junos_command:
host: "{{ inventory_hostname }}"
user: me
ssh_private_key_file: ../mine
port: 22
dest_dir: /home/ansible/juniper-commit-diffs
command: show system rollback 0 compare 1
Expected results output of command saved to file
Actual results
FAILED! => {"changed": false, "module_stderr": "/home/ansible/.ansible/tmp/ansible-tmp-1604074072.5774634-243926931920102/AnsiballZ_juniper_junos_command.py:17: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\n import imp\n/usr/lib/python3/dist-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.\n m.add_string(self.Q_C.public_numbers().encode_point())\n/usr/lib/python3/dist-packages/paramiko/kex_ecdh_nist.py:96: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point\n self.curve, Q_S_bytes\n/usr/lib/python3/dist-packages/paramiko/kex_ecdh_nist.py:111: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.\n hm.add_string(self.Q_C.public_numbers().encode_point())\n/usr/local/lib/python3.7/dist-packages/jnpr/junos/device.py:837: RuntimeWarning: An unknown exception occurred - please report.\n \"An unknown exception occurred - please report.\", RuntimeWarning\nTraceback (most recent call last):\n File \"/home/ansible/.ansible/tmp/ansible-tmp-1604074072.5774634-243926931920102/AnsiballZ_juniper_junos_command.py\", line 113, in <module>\n _ansiballz_main()\n File \"/home/ansible/.ansible/tmp/ansible-tmp-1604074072.5774634-243926931920102/AnsiballZ_juniper_junos_command.py\", line 105, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/home/ansible/.ansible/tmp/ansible-tmp-1604074072.5774634-243926931920102/AnsiballZ_juniper_junos_command.py\", line 48, in invoke_module\n imp.load_module('__main__', mod, module, MOD_DESC)\n File \"/usr/lib/python3.7/imp.py\", line 234, in load_module\n return load_source(name, filename, file)\n File \"/usr/lib/python3.7/imp.py\", line 169, in load_source\n module = _exec(spec, sys.modules[name])\n File \"<frozen importlib._bootstrap>\", line 630, in _exec\n File \"<frozen importlib._bootstrap_external>\", line 728, in exec_module\n File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n File \"/tmp/ansible_juniper_junos_command_payload__rh8996j/__main__.py\", line 520, in <module>\n File \"/tmp/ansible_juniper_junos_command_payload__rh8996j/__main__.py\", line 434, in main\n File \"/usr/local/lib/python3.7/dist-packages/jnpr/junos/rpcmeta.py\", line 387, in __call__\n return self._junos.execute(rpc_cmd, **kvargs)\n File \"/usr/local/lib/python3.7/dist-packages/jnpr/junos/decorators.py\", line 71, in wrapper\n return function(*args, **kwargs)\n File \"/usr/local/lib/python3.7/dist-packages/jnpr/junos/decorators.py\", line 31, in wrapper\n return function(*args, **kwargs)\n File \"/usr/local/lib/python3.7/dist-packages/jnpr/junos/device.py\", line 816, in execute\n filter_xml=kvargs.get(\"filter_xml\"),\n File \"/usr/local/lib/python3.7/dist-packages/jnpr/junos/decorators.py\", line 117, in wrapper\n rsp = function(self, *args, **kwargs)\n File \"/usr/local/lib/python3.7/dist-packages/jnpr/junos/device.py\", line 1419, in _rpc_reply\n return self._conn.rpc(rpc_cmd_e, filter_xml)._NCElement__doc\n File \"/usr/local/lib/python3.7/dist-packages/ncclient/manager.py\", line 231, in execute\n huge_tree=self._huge_tree).request(*args, **kwds)\n File \"/usr/local/lib/python3.7/dist-packages/ncclient/operations/third_party/juniper/rpc.py\", line 52, in request\n return self._request(rpc)\n File \"/usr/local/lib/python3.7/dist-packages/ncclient/operations/rpc.py\", line 350, in _request\n self._reply.parse()\n File \"/usr/local/lib/python3.7/dist-packages/ncclient/operations/rpc.py\", line 160, in parse\n root = self._root = to_ele(self._raw, huge_tree=self._huge_tree) # The <rpc-reply> element\n File \"/usr/local/lib/python3.7/dist-packages/ncclient/xml_.py\", line 124, in to_ele\n return x if etree.iselement(x) else etree.fromstring(x.encode('UTF-8'), parser=_get_parser(huge_tree))\n File \"src/lxml/etree.pyx\", line 3237, in lxml.etree.fromstring\n File \"src/lxml/parser.pxi\", line 1896, in lxml.etree._parseMemoryDocument\n File \"src/lxml/parser.pxi\", line 1784, in lxml.etree._parseDoc\n File \"src/lxml/parser.pxi\", line 1141, in lxml.etree._BaseParser._parseDoc\n File \"src/lxml/parser.pxi\", line 615, in lxml.etree._ParserContext._handleParseResultDoc\n File \"src/lxml/parser.pxi\", line 725, in lxml.etree._handleParseResult\n File \"src/lxml/parser.pxi\", line 654, in lxml.etree._raiseParseError\n File \"<string>\", line 248\nlxml.etree.XMLSyntaxError: StartTag: invalid element name, line 248, column 40\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
on the device cli the command returns the expected output. out of hundreds of devices, 3 are giving this failure at the moment.
Hi @fredtj - Try below playbook.
- name: Print diff between current and rollback 1. No check. No commit.
juniper_junos_config:
rollback: 1
diff: true
check: false
commit: false
register: response
- name: Print the msg.
debug:
var: response
Sample Output :
"response": {
"changed": true,
"diff": {
"prepared": "\n[edit]\n+ interfaces {\n+ ge-0/0/1 {\n+ disable;\n+ }\n+ }\n"
},
"diff_lines": [
"",
"[edit]",
"+ interfaces {",
"+ ge-0/0/1 {",
"+ disable;",
"+ }",
"+ }"
],
"failed": false,
"msg": "Configuration has been: opened, rolled back, diffed, closed.",
}
Refer - https://junos-ansible-modules.readthedocs.io/en/2.4.0/juniper_junos_config.html for similar examples.
Let us know if this works for you.
hello, thanks, i did test using that method, but it is not comparing the configs as i would like - it is comparing the configs the wrong way round (ie the + and - etc are reversed)
@rahkumar651991 i've tracked this problem down, the error occurs when the diff/compare contains an & or < or >
any chance you could take a look please?
the problem line is in library/juniper_junos_command.py
resp = junos_module.dev.rpc(rpc, ignore_warning=ignore_warning, normalize=bool(format == 'xml'))
the traceback is very similar to: https://stackoverflow.com/questions/37005830/how-to-read-an-xml-file-with-sign/37006646
is there any workaround you are aware of?
finally managed to fix this by editing _xml.py in ncclient package and adding recover=True to the XMLParser construct:
parser = etree.XMLParser(recover=True)
huge_parser = etree.XMLParser(recover=True, huge_tree=True)
@property
def tostring(self):
"""return a pretty-printed string output for rpc reply"""
parser = etree.XMLParser(recover=True, remove_blank_text=True, huge_tree=self.__huge_tree)
outputtree = etree.XML(etree.tostring(self.__doc), parser)
return etree.tostring(outputtree, pretty_print=True)
not sure why it is a problem as when i run "show system rollback 0 compare 1 | display xml" on the device, the ampersand character is returned as &
@rahkumar651991 let me know if i should report this elsewhere
@fredtj - thanks for the analysis. Could you help me with the exact output you are seeing for the error. both
- when run manually on device cli
- received at the ncclient/ansible
I am not able to replicate the scenario in my local setup. I will try to hardocode the values and try to execute it.
first, create a commit containing an & character:
me@srx110> show system rollback 0 compare 1
[edit snmp]
- location "Blah Blah, Blah Blah Blah, Blah Blah";
+ location "Blah Blah, Blah Blah Blah, Blah Blah &";
me@srx110> show system rollback 0 compare 1 | display xml
<rpc-reply xmlns:junos="http://xml.juniper.net/junos/12.3X48/junos">
<rollback-information>
<configuration-information>
<configuration-output>
[edit snmp]
- location "Blah Blah, Blah Blah Blah, Blah Blah";
+ location "Blah Blah, Blah Blah Blah, Blah Blah &";
</configuration-output>
</configuration-information>
</rollback-information>
<cli>
<banner></banner>
</cli>
</rpc-reply>
without the & sign in the commit, ansible runs the command "show system rollback 0 compare 1" OK.
With the & sign, here is the error:
/root/.ansible/tmp/ansible-tmp-1606920050.6425843-134013982255409/AnsiballZ_juniper_junos_command.py:17: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
/usr/lib/python3/dist-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
m.add_string(self.Q_C.public_numbers().encode_point())
/usr/lib/python3/dist-packages/paramiko/kex_ecdh_nist.py:96: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
self.curve, Q_S_bytes
/usr/lib/python3/dist-packages/paramiko/kex_ecdh_nist.py:111: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
hm.add_string(self.Q_C.public_numbers().encode_point())
/usr/local/lib/python3.7/dist-packages/jnpr/junos/device.py:837: RuntimeWarning: An unknown exception occurred - please report.
"An unknown exception occurred - please report.", RuntimeWarning
Traceback (most recent call last):
File "/root/.ansible/tmp/ansible-tmp-1606920050.6425843-134013982255409/AnsiballZ_juniper_junos_command.py", line 113, in <module>
_ansiballz_main()
File "/root/.ansible/tmp/ansible-tmp-1606920050.6425843-134013982255409/AnsiballZ_juniper_junos_command.py", line 105, in _ansiballz_main
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
File "/root/.ansible/tmp/ansible-tmp-1606920050.6425843-134013982255409/AnsiballZ_juniper_junos_command.py", line 48, in invoke_module
imp.load_module('__main__', mod, module, MOD_DESC)
File "/usr/lib/python3.7/imp.py", line 234, in load_module
return load_source(name, filename, file)
File "/usr/lib/python3.7/imp.py", line 169, in load_source
module = _exec(spec, sys.modules[name])
File "<frozen importlib._bootstrap>", line 630, in _exec
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/tmp/ansible_juniper_junos_command_payload_gswqsd8b/__main__.py", line 520, in <module>
File "/tmp/ansible_juniper_junos_command_payload_gswqsd8b/__main__.py", line 434, in main
File "/usr/local/lib/python3.7/dist-packages/jnpr/junos/rpcmeta.py", line 387, in __call__
return self._junos.execute(rpc_cmd, **kvargs)
File "/usr/local/lib/python3.7/dist-packages/jnpr/junos/decorators.py", line 71, in wrapper
return function(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/jnpr/junos/decorators.py", line 31, in wrapper
return function(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/jnpr/junos/device.py", line 816, in execute
filter_xml=kvargs.get("filter_xml"),
File "/usr/local/lib/python3.7/dist-packages/jnpr/junos/decorators.py", line 117, in wrapper
rsp = function(self, *args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/jnpr/junos/device.py", line 1419, in _rpc_reply
return self._conn.rpc(rpc_cmd_e, filter_xml)._NCElement__doc
File "/usr/local/lib/python3.7/dist-packages/ncclient/manager.py", line 231, in execute
huge_tree=self._huge_tree).request(*args, **kwds)
File "/usr/local/lib/python3.7/dist-packages/ncclient/operations/third_party/juniper/rpc.py", line 52, in request
return self._request(rpc)
File "/usr/local/lib/python3.7/dist-packages/ncclient/operations/rpc.py", line 350, in _request
self._reply.parse()
File "/usr/local/lib/python3.7/dist-packages/ncclient/operations/rpc.py", line 160, in parse
root = self._root = to_ele(self._raw, huge_tree=self._huge_tree) # The <rpc-reply> element
File "/usr/local/lib/python3.7/dist-packages/ncclient/xml_.py", line 124, in to_ele
return x if etree.iselement(x) else etree.fromstring(x.encode('UTF-8'), parser=_get_parser(huge_tree))
File "src/lxml/etree.pyx", line 3237, in lxml.etree.fromstring
File "src/lxml/parser.pxi", line 1896, in lxml.etree._parseMemoryDocument
File "src/lxml/parser.pxi", line 1784, in lxml.etree._parseDoc
File "src/lxml/parser.pxi", line 1141, in lxml.etree._BaseParser._parseDoc
File "src/lxml/parser.pxi", line 615, in lxml.etree._ParserContext._handleParseResultDoc
File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
File "<string>", line 4
lxml.etree.XMLSyntaxError: xmlParseEntityRef: no name, line 4, column 70
this is also happening with < and > in the commit, so failures occur for things like the following:
set policy-options prefix-list ntp-servers apply-path "system ntp server <*>"