ansible-junos-stdlib
ansible-junos-stdlib copied to clipboard
Juniper
juniper_junos_command missing data, json duplicate keys not converted to list
Issue Type
- Bug Report
Module Name
juniper_junos_command
juniper.device collection and Python libraries version
ansible --version
ansible 2.10.8
config file = /Users/xxx/git/ansible-scripts/ansible.cfg
configured module search path = ['/Users/xxx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Users/xxx/Library/Python/3.8/lib/python/site-packages/ansible
executable location = /Users/xxx/Library/Python/3.8/bin/ansible
python version = 3.8.2 (default, Jun 8 2021, 11:59:35) [Clang 12.0.5 (clang-1205.0.22.11)]
pip freeze
ansible==2.10.7
ansible-base==2.10.8
ansible-lint==4.3.7
bcrypt==3.2.0
Cerberus==1.3.2
certifi==2020.6.20
cffi==1.14.3
chardet==3.0.4
click==7.1.2
colorama==0.4.4
commonmark==0.9.1
cryptography==3.1.1
Flask==1.1.2
flask-swagger-ui==3.36.0
future==0.18.2
gunicorn==20.1.0
idna==2.10
ipaddr==2.2.0
itsdangerous==1.1.0
Jinja2==2.11.2
junos-eznc==2.6.2
jxmlease==1.0.3
lxml==4.5.2
MarkupSafe==1.1.1
mysql-connector-python==8.0.22
ncclient==0.6.9
netaddr==0.8.0
ntc-templates==1.5.0
numpy==1.20.3
packaging==20.9
pandas==1.2.4
paramiko==2.7.2
pathspec==0.8.1
prometheus-client==0.9.0
protobuf==3.14.0
pyang==2.3.2
pycparser==2.20
Pygments==2.7.3
PyNaCl==1.4.0
pyparsing==2.4.7
pyserial==3.4
python-dateutil==2.8.1
pytricia==1.0.2
pytz==2021.1
PyYAML==5.3.1
requests==2.24.0
rich==9.6.1
ruamel.yaml==0.16.12
ruamel.yaml.clib==0.2.2
scp==0.13.2
six==1.15.0
termcolor==1.1.0
textfsm==1.1.0
transitions==0.8.3
typing-extensions==3.7.4.3
urllib3==1.25.10
Werkzeug==1.0.1
xmltodict==0.12.0
yamllint==1.25.0
yamlordereddictloader==0.4.0
OS / Environment
Model: ptx10008 Junos: 17.3R3-S12.3
Summary
When running the command "show bgp neighbor" and asking for json output the output returns duplicate keys (addpath-send and addpath-recieve are used multiple times). The json retrieved by ansible omits the duplicate keys and we loose data... the json returned is incomplete. Python does not permitted duplicate keys in a dictionary, so only the last entry will survive.
If I take a "show bgp neighbor | display json" from the CLI directly, its possible to write my own json.loads function to turn the duplicate keys into an array (as discussed in this link and other places), but the ansible parsing module would have to updated in a similar way to avoid loosing output.
https://stackoverflow.com/questions/24416960/convert-json-object-with-duplicate-keys-to-json-array
Steps to reproduce
You can see the issue on the router CLI... the json keys are duplicated. I guess this is technically not an RFC violation, but it would seem that most conversion utilities (python) will not allow this.
show bgp neighbor | display json
<snip>
"addpath-send" : [
{
"nlri-type" : [
{
"data" : "inet-unicast"
}
],
"addpath-send-type" : [
{
"data" : "Specified number of paths"
}
],
"addpath-send-count" : [
{
"data" : "2"
}
]
}
],
"addpath-receive" : [
{
"nlri-type" : [
{
"data" : "inet-unicast"
}
],
"addpath-receive-type" : [
{
"data" : "enabled"
}
]
}
],
"addpath-send" : [
{
"nlri-type" : [
{
"data" : "inet6-labeled-unicast"
}
<snip>
When the playbook is run... the resulting output will only show the last entry in this case inet6-labeled-unicast, the inet-unicast data is lost.
TASK [debug return data] *******************************************************
ok: [edge10.nso5] => {
"msg": {
"changed": false,
"command": "show bgp neighbor",
"failed": false,
"format": "json",
"msg": "The command executed successfully.",
"parsed_output": {
"bgp-information": [
{
<snip>
"bgp-option-information": [
{
"addpath-receive": [
{
"addpath-receive-type": [
{
"data": "enabled"
}
],
"nlri-type": [
{
"data": "inet6-labeled-unicast"
}
]
}
],
"addpath-send": [
{
"addpath-send-count": [
{
"data": "2"
}
],
"addpath-send-type": [
{
"data": "Specified number of paths"
}
],
"nlri-type": [
{
"data": "inet6-labeled-unicast"
}
]
}
],
- hosts: "{{device}}"
gather_facts: no
connection: local
roles:
- juniper.junos
- name: get junos bgp neighbor data
juniper_junos_command:
commands:
- show bgp neighbor
display: json
register: show_bgp_neighbor
when: os == "junos"
- name: debug return data
debug:
msg: "{{ show_bgp_neighbor }}"
Expected results
The duplicate keys from the juniper must converted to ensure data is not lost. The juniper module needs to take the duplicate keys from the raw router json and convert them into a list (or the duplicates would have to be eliminated in junos). I assume the juniper ansible module is using json.loads or something similar, by default, json.loads will overwrite the duplicate keys so only the last entry will survive.
The ansible moudles must return something like this in the ansible register to avoid loosing parts of the output and ensure the playbook has the full contents of the command from the router.
"addpath-send": [
{
"nlri-type": "inet-unicast",
"addpath-send-type": "Specified number of paths",
"addpath-send-count": "2"
},
{
"nlri-type": "inet6-labeled-unicast",
"addpath-send-type": "Specified number of paths",
"addpath-send-count": "2"
}
],
"addpath-receive": [
{
"nlri-type": "inet-unicast",
"addpath-receive-type": "enabled"
},
{
"nlri-type": "inet6-labeled-unicast",
"addpath-receive-type": "enabled"
}
],
Actual results
juniper_junos_command is returning incomplete data from the router when the display type is json.
"bgp-option-information": [
{
"addpath-receive": [
{
"addpath-receive-type": [
{
"data": "enabled"
}
],
"nlri-type": [
{
"data": "inet6-labeled-unicast"
}
]
}
],
"addpath-send": [
{
"addpath-send-count": [
{
"data": "2"
}
],
"addpath-send-type": [
{
"data": "Specified number of paths"
}
],
"nlri-type": [
{
"data": "inet6-labeled-unicast"
}
]
}
],
