juniper.device.software - Software is downloaded for each RE when using remote_package

[jnicholson56]

Issue Type

• Bug Report
• Feature Idea

Module Name

juniper.device collection

Collection        Version
----------------- -------
juniper.device    1.0.2

ansible [core 2.11.12]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/josephn/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.7/dist-packages/ansible
  ansible collection location = /home/josephn/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.7.3 (default, Jun 29 2023, 18:03:57) [GCC 8.3.0]
  jinja version = 2.11.2
  libyaml = True

aggregate6==1.0.12
ansible==4.10.0
ansible-core==2.11.12
ansible-lint==5.4.0
ansible-runner==2.2.1
apache-libcloud==2.4.0
appdirs==1.4.4
asciitree==0.3.3
asn1crypto==0.24.0
asyncpg==0.20.1
atlassian-python-api==3.36.0
atomicwrites==1.1.5
attrs==19.3.0
backcall==0.2.0
backports.cached-property==1.0.1
badmeester==0.7
bcrypt==3.1.6
beautifulsoup4==4.7.1
black==22.3.0
bracex==2.3.post1
build==0.7.0
certifi==2018.8.24
cffi==1.12.2
cfgv==3.1.0
chardet==3.0.4
click==8.0.3
colorama==0.4.6
commonmark==0.9.1
confluent-kafka==0.11.6
coverage==5.2
cryptography==2.6.1
deadmanswitch==0.1.0
debugpy==1.6.6
decorator==5.1.1
deepmerge==1.0.1
Deprecated==1.2.13
distlib==0.3.1
distro-info==0.21
docutils==0.19
dropbox==11.33.0
enrich==1.2.7
entrypoints==0.3
et-xmlfile==1.1.0
fail2ban==0.10.2
filelock==3.0.12
future==0.18.2
goodconf==1.0.0
greenlet==0.4.15
html5lib==1.0.1
httplib2==0.11.3
identify==1.4.21
idna==2.6
importlib-metadata==1.7.0
ipykernel==6.16.2
ipython==7.34.0
irrtree==1.4.0
isc==2.0
jedi==0.18.2
Jinja2==2.11.2
jmespath==0.9.4
junos-eznc==2.6.6
jupyter_client==7.4.9
jupyter_core==4.12.0
keyring==17.1.1
keyrings.alt==3.1.1
lockfile==0.12.2
loguru==0.5.3
lxml==4.6.5
MarkupSafe==1.1.1
matplotlib-inline==0.1.6
mkcfg==1.15.1
mock==4.0.2
more-itertools==8.4.0
msgpack==0.5.6
mypy-extensions==0.4.3
mysqlclient==1.3.10
ncclient==0.6.13
neovim==0.3.0
nest-asyncio==1.5.6
netaddr==0.7.19
netconf-console2==3.0.0
netmiko==4.1.2
nodeenv==1.4.0
ntc-templates==3.2.0
ntlm-auth==1.1.0
nttslack==0.1.0
nttsplunk==0.1
numpy==1.21.6
oauthlib==3.2.0
openpyxl==3.0.10
packaging==20.4
pandas==1.3.5
paramiko==2.7.2
parso==0.8.3
passlib==1.7.1
pathspec==0.9.0
peewee==3.13.3
pep517==0.12.0
pexpect==4.8.0
pickleshare==0.7.5
pipenv==2020.6.2
platformdirs==2.5.2
pluggy==0.13.1
ply==3.11
pre-commit==2.6.0
prefix-list-utils==0.8.5
progressbar2==3.34.3
prompt-toolkit==3.0.38
psutil==5.9.4
psycopg2==2.8.5
ptyprocess==0.6.0
py==1.9.0
py-dateutil==2.2
py-radix==0.10.0
py-spy==0.3.14
pyasn1==0.4.2
pybgpstream==2.0.2
pyconfigtool==6.2.15.0
pycparser==2.19
pycrypto==2.6.1
pycryptodomex==3.6.1
pydantic==1.8.2
pydruid==0.6.2
Pygments==2.12.0
PyGObject==3.30.4
pygtrie==2.3.3
pyinotify==0.9.6
pykerberos==1.1.14
PyNaCl==1.3.0
pyparsing==2.4.7
pyserial==3.4
pysftp==0.2.9
pysmi==0.3.2
pysnmp==4.4.6
pysros==22.10.1
pytest==5.4.3
pytest-cov==2.10.0
python-apt==1.8.4.3
python-daemon==2.3.2
python-dateutil==2.8.2
python-utils==2.2.0
pytz==2022.1
pywinrm==0.3.0
pyxdg==0.25
PyYAML==5.4.1
pyzmq==25.0.1
requests==2.21.0
requests-kerberos==0.11.0
requests-ntlm==1.1.0
requests-oauthlib==1.3.1
resolvelib==0.5.5
rich==12.4.4
rpki-ov-checker==0.0.5
rtrsub==1.0.0
ruamel.yaml==0.16.10
ruamel.yaml.clib==0.2.0
scp==0.13.3
scrypt==0.8.0
SecretStorage==2.3.1
setproctitle==1.2.2
simplejson==3.17.0
six==1.12.0
slack==0.0.2
soupsieve==1.8
SQLAlchemy==1.3.18
stone==3.3.1
structlog==22.3.0
systemd-python==234
telescent==0.1.0
telescent-automate==0.1.5
telescent-cli==0.1.1
tenacity==6.2.0
textfsm==1.1.2
toml==0.10.1
tomli==2.0.1
tornado==6.2
tqdm==4.64.0
traitlets==5.9.0
transitions==0.9.0
typed-ast==1.5.3
typer==0.4.0
typing==3.7.4.3
typing_extensions==4.2.0
unattended-upgrades==0.1
urllib3==1.24.1
virtualenv==20.0.26
virtualenv-clone==0.5.4
wcmatch==8.4.1
wcwidth==0.2.5
webencodings==0.5.1
wrapt==1.15.0
xmltodict==0.12.0
yamllint==1.15.0
yamlordereddictloader==0.4.0
zipp==3.1.0

OS / Environment

Tested on the following versions: 20.2.R3-S3.6 20.2.R3.9 20.2.R3-S2.5 22.2R3.15

Not specific to any version

Summary

When upgrading a router and designating a remote_package, Ansible initiates a file transfer for each RE independently instead of downloading once and installing on both RE's.

For RE0, the package is downloaded to the primary RE and staged to RE0. For RE1, the package is downloaded to the primary RE and started to RE1.

This happens if RE0 or RE1 is primary.

SW packages can be large and take quite some time to download depending on how far away the file server is or what version of JUNOS is being used. Duplicating downloads can double this time and cause upgrades to take too long to complete.

Steps to reproduce

Ran playbook with the following upgrade task.

    - name: Upgrade Router
      juniper.device.software:
        version: "20.2R3.9"
        remote_package: "ftp://XXX.XXX.XXX.XXX:/junos-vmhost-install-mx-x86-64-20.2R3.9.tgz"
        install_timeout: 3600
        vmhost: True
        cleanfs: False
        validate: False
        reboot: True
        level: INFO
        logfile: "{{ logfile }}"

Expected results

The package should be downloaded once and stage to both RE's versus a download per RE.

Actual results

2023-08-02 18:11:30,180 - jnpr.ansible_module.juniper.device.software - INFO - [router_name_sanitized] installing software on RE0 ... please be patient ...
2023-08-02 18:11:30,181 - ncclient.operations.rpc - INFO - [host router_name_sanitized session-id 39741] Requesting 'ExecuteRpc'
2023-08-02 18:11:30,280 - ncclient.transport.ssh - INFO - [host router_name_sanitized session-id 39741] Sending:
b'<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f028af49-0911-4435-82bb-66fa8e0c3a81"><request-vmhost-package-add><package-name>ftp://XXX.XXX.XXX.XXX:/jnx/old/mx10k/junos-vmhost-install-mx-x86-64-20.2R3.9.tgz</package-name><re0/><no-validate/></request-vmhost-package-add></nc:rpc>]]>]]>'
tail: inotify resources exhausted
tail: inotify cannot be used, reverting to polling
2023-08-02 18:19:06,781 - ncclient.transport.ssh - INFO - [host router_name_sanitized session-id 39741] Received message from host
2023-08-02 18:19:06,783 - jnpr.ansible_module.juniper.device.software - INFO - [router_name_sanitized] software pkgadd package-result: 0
Output:
Fetching package...

Package downloaded to /var/tmp/...transferring.file.........sIQHBV/junos-vmhost-install-mx-x86-64-20.2R3.9.tgz


2023-08-02 18:19:06,784 - jnpr.ansible_module.juniper.device.software - INFO - [router_name_sanitized] installing software on RE1 ... please be patient ...
2023-08-02 18:19:06,784 - ncclient.operations.rpc - INFO - [host router_name_sanitized session-id 39741] Requesting 'ExecuteRpc'
2023-08-02 18:19:06,882 - ncclient.transport.ssh - INFO - [host router_name_sanitized session-id 39741] Sending:
b'<?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:cb0694ad-b2d4-4094-ae65-ff2178e314d0"><request-vmhost-package-add><package-name>ftp://XXX.XXX.XXX.XXX:/jnx/old/mx10k/junos-vmhost-install-mx-x86-64-20.2R3.9.tgz</package-name><re1/><no-validate/></request-vmhost-package-add></nc:rpc>]]>]]>'
2023-08-02 18:27:24,279 - ncclient.transport.ssh - INFO - [host router_name_sanitized session-id 39741] Received message from host
2023-08-02 18:27:24,281 - jnpr.ansible_module.juniper.device.software - INFO - [router_name_sanitized] software pkgadd package-result: 0
Output:
Fetching package...

Package downloaded to /var/tmp/...transferring.file.........JP8jZ5/junos-vmhost-install-mx-x86-64-20.2R3.9.tgz

Pushing /var/tmp/...transferring.file.........JP8jZ5/junos-vmhost-install-mx-x86-64-20.2R3.9.tgz to re1:/var/tmp/junos-vmhost-install-mx-x86-64-20.2R3.9.tgz

[apurvaraghu]

Is this specific to vmhost device or any Junos device?

[jnicholson56]

I have only tested it on vmhost devices. I don't remember anything in the underlying code that was specific to vmhost. I would expect this behavior on any junos device.

On Tue, Feb 13, 2024, 13:49 apurvaraghu @.***> wrote:

Is this specific to vmhost device or any Junos device?

— Reply to this email directly, view it on GitHub https://github.com/Juniper/ansible-junos-stdlib/issues/624#issuecomment-1942180960, or unsubscribe https://github.com/notifications/unsubscribe-auth/AILVUEVJSZXCSOKGM5IUYBLYTOYSXAVCNFSM6AAAAAA3BY2ZEWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBSGE4DAOJWGA . You are receiving this because you authored the thread.Message ID: @.***>