sonic-utilities
sonic-utilities copied to clipboard
Published
20 hours ago •
sonic-net
sonic-net
Add Multi ASIC support for apply-patch
What I did
Add Multi-ASIC GCU support apply-patch.
How I did it
- Categorize configuration as JSON patch format per ASIC.
- Apply patch per ASIC, including localhost.
How to verify it
admin@str3-7800-lc3-1:~/gcu$ cat jsonpatch.json
[
{
"op": "add",
"path": "/asic0/PORTCHANNEL/PortChannel108/admin_status",
"value": "down"
},
{
"op": "replace",
"path": "/localhost/BGP_DEVICE_GLOBAL/STATE/tsa_enabled",
"value": "true"
},
{
"op": "replace",
"path": "/asic0/BGP_DEVICE_GLOBAL/STATE/tsa_enabled",
"value": "true"
},
{
"op": "replace",
"path": "/asic1/BGP_DEVICE_GLOBAL/STATE/tsa_enabled",
"value": "true"
}
]
admin@str3-7800-lc3-1:~/gcu$ sudo config apply-patch jsonpatch.json
Patch: [{"op": "add", "path": "/asic0/PORTCHANNEL/PortChannel108/admin_status", "value": "down"}, {"op": "replace", "path": "/localhost/BGP_DEVICE_GLOBAL/STATE/tsa_enabled", "value": "true"}, {"op": "replace", "path": "/asic0/BGP_DEVICE_GLOBAL/STATE/tsa_enabled", "value": "true"}, {"op": "replace", "path": "/asic1/BGP_DEVICE_GLOBAL/STATE/tsa_enabled", "value": "true"}]
('asic0', [{'op': 'add', 'path': '/PORTCHANNEL/PortChannel108/admin_status', 'value': 'down'}, {'op': 'replace', 'path': '/BGP_DEVICE_GLOBAL/STATE/tsa_enabled', 'value': 'true'}])
NameSpace: asic0
generic_update_factory: <generic_config_updater.generic_updater.GenericUpdateFactory object at 0x7f61ae2a18e0>
generic_update_factory: <generic_config_updater.generic_updater.GenericUpdateFactory object at 0x7f61ae2a18e0>
Patch Applier: asic0: Patch application starting.
Patch Applier: asic0: Patch: [{"op": "add", "path": "/PORTCHANNEL/PortChannel108/admin_status", "value": "down"}, {"op": "replace", "path": "/BGP_DEVICE_GLOBAL/STATE/tsa_enabled", "value": "true"}]
Patch Applier: Getting current config db.
Patch Applier: Simulating the target full config after applying the patch.
Patch Applier: Validating all JsonPatch operations are permitted on the specified fields
Patch Applier: Validating target config does not have empty tables, since they do not show up in ConfigDb.
Patch Applier: Sorting patch updates.
<class 'generic_config_updater.patch_sorter.StrictPatchSorter'>
Patch Applier: The patch was sorted into 0 changes.
Patch Applier: Applying 0 changes in order.
Patch Applier: Verifying patch updates are reflected on ConfigDB.
Patch Applier: Patch application completed.
('localhost', [{'op': 'replace', 'path': '/BGP_DEVICE_GLOBAL/STATE/tsa_enabled', 'value': 'true'}])
NameSpace:
generic_update_factory: <generic_config_updater.generic_updater.GenericUpdateFactory object at 0x7f61b1b547c0>
generic_update_factory: <generic_config_updater.generic_updater.GenericUpdateFactory object at 0x7f61b1b547c0>
Patch Applier: : Patch application starting.
Patch Applier: : Patch: [{"op": "replace", "path": "/BGP_DEVICE_GLOBAL/STATE/tsa_enabled", "value": "true"}]
Patch Applier: Getting current config db.
Patch Applier: Simulating the target full config after applying the patch.
Patch Applier: Validating all JsonPatch operations are permitted on the specified fields
Patch Applier: Validating target config does not have empty tables, since they do not show up in ConfigDb.
Patch Applier: Sorting patch updates.
<class 'generic_config_updater.patch_sorter.StrictPatchSorter'>
Patch Applier: The patch was sorted into 0 changes.
Patch Applier: Applying 0 changes in order.
Patch Applier: Verifying patch updates are reflected on ConfigDB.
Patch Applier: Patch application completed.
('asic1', [{'op': 'replace', 'path': '/BGP_DEVICE_GLOBAL/STATE/tsa_enabled', 'value': 'true'}])
NameSpace: asic1
generic_update_factory: <generic_config_updater.generic_updater.GenericUpdateFactory object at 0x7f61b1b547c0>
generic_update_factory: <generic_config_updater.generic_updater.GenericUpdateFactory object at 0x7f61b1b547c0>
Patch Applier: asic1: Patch application starting.
Patch Applier: asic1: Patch: [{"op": "replace", "path": "/BGP_DEVICE_GLOBAL/STATE/tsa_enabled", "value": "true"}]
Patch Applier: Getting current config db.
Patch Applier: Simulating the target full config after applying the patch.
Patch Applier: Validating all JsonPatch operations are permitted on the specified fields
Patch Applier: Validating target config does not have empty tables, since they do not show up in ConfigDb.
Patch Applier: Sorting patch updates.
<class 'generic_config_updater.patch_sorter.StrictPatchSorter'>
Patch Applier: The patch was sorted into 0 changes.
Patch Applier: Applying 0 changes in order.
Patch Applier: Verifying patch updates are reflected on ConfigDB.
Patch Applier: Patch application completed.
Patch applied successfully.
Failed case
admin@str3-7800-lc3-1:~/gcu$ sudo config apply-patch jsonpatch.json
Patch: [{"op": "add", "path": "/asic0/PORTCHANNEL/PortChannel108/admin_status", "value": "down"}, {"op": "replace", "path": "/localhost/BGP_DEVICE_GLOBAL/STATE/tsa_enabled", "value": "true"}, {"op": "replace", "path": "/asic0/BGP_DEVICE_GLOBAL/STATE/tsa_enabled", "value": "true"}, {"op": "replace", "path": "/asic1/BGP_DEVICE_GLOBAL/STATE/tsa_enabled", "value": "true"}]
('asic0', [{'op': 'add', 'path': '/PORTCHANNEL/PortChannel108/admin_status', 'value': 'down'}, {'op': 'replace', 'path': '/BGP_DEVICE_GLOBAL/STATE/tsa_enabled', 'value': 'true'}])
NamSpace: asic0
generic_update_factory: <generic_config_updater.generic_updater.GenericUpdateFactory object at 0x7f6f1e3bc7c0>
generic_update_factory: <generic_config_updater.generic_updater.GenericUpdateFactory object at 0x7f6f1e3bc7c0>
Patch Applier: asic0: Patch application starting.
Patch Applier: asic0: Patch: [{"op": "add", "path": "/PORTCHANNEL/PortChannel108/admin_status", "value": "down"}, {"op": "replace", "path": "/BGP_DEVICE_GLOBAL/STATE/tsa_enabled", "value": "true"}]
Patch Applier: Getting current config db.
Patch Applier: Simulating the target full config after applying the patch.
Patch Applier: Validating all JsonPatch operations are permitted on the specified fields
Patch Applier: Validating target config does not have empty tables, since they do not show up in ConfigDb.
Patch Applier: Sorting patch updates.
<class 'generic_config_updater.patch_sorter.StrictPatchSorter'>
path: /sonic-bgp-device-global:sonic-bgp-device-global/BGP_DEVICE_GLOBAL/STATE/tsa_enabled
Failed to apply patch due to: 'NoneType' object has no attribute 'find_path'
Usage: config apply-patch [OPTIONS] PATCH_FILE_PATH
Try "config apply-patch -h" for help.
Error: 'NoneType' object has no attribute 'find_path'
Previous command output (if the output of a command-line utility has changed)
New command output (if the output of a command-line utility has changed)
The old PR's conversation is here: https://github.com/sonic-net/sonic-utilities/pull/3219, the old PR was mistakenly deleting commits. #Resolved
class ChangeApplier:
ChangeApplier could add a member self.config_db for the ConfigDBConnector. and rename existing self.config_db to self.running_config #Closed
Refers to: generic_config_updater/change_applier.py:73 in 60bbcfe. [](commit_id = 60bbcfe5d1a393cc42a183d7930721d6a9018bf8, deletion_comment = False)
class ChangeApplier:
This is real db connector object, please ref this function
def set_config(config_db, tbl, key, data):
config_db.set_entry(tbl, key, data)
In reply to: 2037887648
Refers to: generic_config_updater/change_applier.py:73 in 60bbcfe. [](commit_id = 60bbcfe5d1a393cc42a183d7930721d6a9018bf8, deletion_comment = False)
Hi @xincunli-sonic , I think of this after todays meeting. Single asic can utilize empty patch for YANG validation, can multi-asic also support empty patch validation?
admin@bjw-can-7260-11:~$ cat empty.json
[]
admin@bjw-can-7260-11:~$ sudo config apply-patch empty.json
Patch Applier: Patch application starting.
Patch Applier: Patch: []
Patch Applier: Getting current config db.
Patch Applier: Simulating the target full config after applying the patch.
Patch Applier: Validating all JsonPatch operations are permitted on the specified fields
Patch Applier: Validating target config does not have empty tables, since they do not show up in ConfigDb.
Patch Applier: Sorting patch updates.
Patch Applier: The patch was converted into 0 changes.
Patch Applier: Applying 0 changes in order.
Patch Applier: Verifying patch updates are reflected on ConfigDB.
Patch Applier: Patch application completed.
Patch applied successfully.
``` #Resolved
Hi @xincunli-sonic , I think of this after todays meeting. Single asic can utilize empty patch for YANG validation, can multi-asic also support empty patch validation?
admin@bjw-can-7260-11:~$ cat empty.json [] admin@bjw-can-7260-11:~$ sudo config apply-patch empty.json Patch Applier: Patch application starting. Patch Applier: Patch: [] Patch Applier: Getting current config db. Patch Applier: Simulating the target full config after applying the patch. Patch Applier: Validating all JsonPatch operations are permitted on the specified fields Patch Applier: Validating target config does not have empty tables, since they do not show up in ConfigDb. Patch Applier: Sorting patch updates. Patch Applier: The patch was converted into 0 changes. Patch Applier: Applying 0 changes in order. Patch Applier: Verifying patch updates are reflected on ConfigDB. Patch Applier: Patch application completed. Patch applied successfully.
Hi @xincunli-sonic , I think of this after todays meeting. Single asic can utilize empty patch for YANG validation, can multi-asic also support empty patch validation?
admin@bjw-can-7260-11:~$ cat empty.json [] admin@bjw-can-7260-11:~$ sudo config apply-patch empty.json Patch Applier: Patch application starting. Patch Applier: Patch: [] Patch Applier: Getting current config db. Patch Applier: Simulating the target full config after applying the patch. Patch Applier: Validating all JsonPatch operations are permitted on the specified fields Patch Applier: Validating target config does not have empty tables, since they do not show up in ConfigDb. Patch Applier: Sorting patch updates. Patch Applier: The patch was converted into 0 changes. Patch Applier: Applying 0 changes in order. Patch Applier: Verifying patch updates are reflected on ConfigDB. Patch Applier: Patch application completed. Patch applied successfully.
Thanks, Good point, added empty case handle which can drive YANG validation for host and all ASICs. #Resolved
Hello， @xincunli-sonic， I'm working on the unit test codes development for this PR. Now I couldn't compile an image including your modification. The error is below. Could you help provide some hints on this? And what's the sonic-buildimage's URL and branch that you are using while developing and testing this PR? Thanks.
Using /sonic/src/sonic-utilities/.eggs/responses-0.25.0-py3.9.egg
running egg_info
writing sonic_utilities.egg-info/PKG-INFO
writing dependency_links to sonic_utilities.egg-info/dependency_links.txt
writing entry points to sonic_utilities.egg-info/entry_points.txt
writing requirements to sonic_utilities.egg-info/requires.txt
writing top-level names to sonic_utilities.egg-info/top_level.txt
reading manifest file 'sonic_utilities.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*' under directory 'data'
writing manifest file 'sonic_utilities.egg-info/SOURCES.txt'
running build_ext
ImportError while loading conftest '/sonic/src/sonic-utilities/tests/conftest.py'.
tests/conftest.py:23: in <module>
import config.main as config
config/main.py:28: in <module>
from sonic_py_common.general import getstatusoutput_noshell
E ImportError: cannot import name 'getstatusoutput_noshell' from 'sonic_py_common.general' (/var/jumao/.local/lib/python3.9/site-packages/sonic_py_common/general.py)
[ FAIL LOG END ] [ target/python-wheels/bullseye/sonic_utilities-1.2-py3-none-any.whl ]
make: *** [slave.mk:766: target/python-wheels/bullseye/sonic_utilities-1.2-py3-none-any.whl] Error 1
make[1]: *** [Makefile.work:448: target/sonic-broadcom.bin] Error 2
make[1]: Leaving directory '/home/jumao/m/msft-2205-ndk-20240402/sonic-buildimage'
make: *** [Makefile:33: target/sonic-broadcom.bin] Error 2
``` #Resolved
Hello， @xincunli-sonic， I'm working on the unit test codes development for this PR. Now I couldn't compile an image including your modification. The error is below. Could you help provide some hints on this? And what's the sonic-buildimage's URL and branch that you are using while developing and testing this PR? Thanks.
Using /sonic/src/sonic-utilities/.eggs/responses-0.25.0-py3.9.egg running egg_info writing sonic_utilities.egg-info/PKG-INFO writing dependency_links to sonic_utilities.egg-info/dependency_links.txt writing entry points to sonic_utilities.egg-info/entry_points.txt writing requirements to sonic_utilities.egg-info/requires.txt writing top-level names to sonic_utilities.egg-info/top_level.txt reading manifest file 'sonic_utilities.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching '*' under directory 'data' writing manifest file 'sonic_utilities.egg-info/SOURCES.txt' running build_ext ImportError while loading conftest '/sonic/src/sonic-utilities/tests/conftest.py'. tests/conftest.py:23: in <module> import config.main as config config/main.py:28: in <module> from sonic_py_common.general import getstatusoutput_noshell E ImportError: cannot import name 'getstatusoutput_noshell' from 'sonic_py_common.general' (/var/jumao/.local/lib/python3.9/site-packages/sonic_py_common/general.py) [ FAIL LOG END ] [ target/python-wheels/bullseye/sonic_utilities-1.2-py3-none-any.whl ] make: *** [slave.mk:766: target/python-wheels/bullseye/sonic_utilities-1.2-py3-none-any.whl] Error 1 make[1]: *** [Makefile.work:448: target/sonic-broadcom.bin] Error 2 make[1]: Leaving directory '/home/jumao/m/msft-2205-ndk-20240402/sonic-buildimage' make: *** [Makefile:33: target/sonic-broadcom.bin] Error 2
What I forked the branch is an old version, if you want to test, please do some manual work to align with the latest change.
There is a problem: the command "config apply-patch" caused some containers to exit.
admin@ixre-egl-board40:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
baeabdcd38a9 70dd58e31088 "/usr/local/bin/supe…" 2 hours ago Up 2 minutes macsec0
bfe465afc0aa docker-router-advertiser:latest "/usr/bin/docker-ini…" 2 hours ago Up 2 minutes radv
b710a81b05b9 70dd58e31088 "/usr/local/bin/supe…" 2 hours ago Up 2 minutes macsec1
3a8ddbc204ad docker-fpm-frr:latest "/usr/bin/docker_ini…" 2 hours ago Up 2 minutes bgp0
e553479d15b1 docker-fpm-frr:latest "/usr/bin/docker_ini…" 2 hours ago Up 2 minutes bgp1
4e35cf0f962f docker-syncd-brcm-dnx:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes syncd0
14f9109c93cc docker-teamd:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes teamd0
f77fd9c53cd7 docker-syncd-brcm-dnx:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes syncd1
57209995a85b docker-teamd:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes teamd1
fd030291d9fa docker-orchagent:latest "/usr/bin/docker-ini…" 2 hours ago Up 2 minutes swss1
cc0377006913 docker-eventd:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes eventd
ae7c871d4e37 docker-database:latest "/usr/local/bin/dock…" 2 hours ago Up 2 minutes database1
aeb234180f94 docker-database:latest "/usr/local/bin/dock…" 2 hours ago Up 2 minutes database0
f0ed626a8152 docker-database:latest "/usr/local/bin/dock…" 2 hours ago Up 2 minutes database
admin@ixre-egl-board40:~$ cat empty.json
[]
admin@ixre-egl-board40:~$ sudo config apply-patch empty.json -d
** DRY RUN EXECUTION **
Patch Applier: Patch application starting.
Patch Applier: Patch: []
Patch Applier: Getting current config db.
Patch Applier: Simulating the target full config after applying the patch.
Patch Applier: Validating all JsonPatch operations are permitted on the specified fields
Patch Applier: Validating target config does not have empty tables, since they do not show up in ConfigDb.
Patch Applier: Sorting patch updates.
Patch Applier: The patch was converted into 0 changes.
Patch Applier: Applying 0 changes in order.
Patch Applier: Verifying patch updates are reflected on ConfigDB.
Patch Applier: Patch application completed.
Patch applied successfully.
admin@ixre-egl-board40:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3a8ddbc204ad docker-fpm-frr:latest "/usr/bin/docker_ini…" 2 hours ago Up 2 minutes bgp0
e553479d15b1 docker-fpm-frr:latest "/usr/bin/docker_ini…" 2 hours ago Up 2 minutes bgp1
4e35cf0f962f docker-syncd-brcm-dnx:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes syncd0
f77fd9c53cd7 docker-syncd-brcm-dnx:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes syncd1
cc0377006913 docker-eventd:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes eventd
ae7c871d4e37 docker-database:latest "/usr/local/bin/dock…" 2 hours ago Up 2 minutes database1
aeb234180f94 docker-database:latest "/usr/local/bin/dock…" 2 hours ago Up 2 minutes database0
f0ed626a8152 docker-database:latest "/usr/local/bin/dock…" 2 hours ago Up 2 minutes database
admin@ixre-egl-board40:~$
There is a problem: the command "config apply-patch" caused some containers to exit.
admin@ixre-egl-board40:~$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES baeabdcd38a9 70dd58e31088 "/usr/local/bin/supe…" 2 hours ago Up 2 minutes macsec0 bfe465afc0aa docker-router-advertiser:latest "/usr/bin/docker-ini…" 2 hours ago Up 2 minutes radv b710a81b05b9 70dd58e31088 "/usr/local/bin/supe…" 2 hours ago Up 2 minutes macsec1 3a8ddbc204ad docker-fpm-frr:latest "/usr/bin/docker_ini…" 2 hours ago Up 2 minutes bgp0 e553479d15b1 docker-fpm-frr:latest "/usr/bin/docker_ini…" 2 hours ago Up 2 minutes bgp1 4e35cf0f962f docker-syncd-brcm-dnx:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes syncd0 14f9109c93cc docker-teamd:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes teamd0 f77fd9c53cd7 docker-syncd-brcm-dnx:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes syncd1 57209995a85b docker-teamd:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes teamd1 fd030291d9fa docker-orchagent:latest "/usr/bin/docker-ini…" 2 hours ago Up 2 minutes swss1 cc0377006913 docker-eventd:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes eventd ae7c871d4e37 docker-database:latest "/usr/local/bin/dock…" 2 hours ago Up 2 minutes database1 aeb234180f94 docker-database:latest "/usr/local/bin/dock…" 2 hours ago Up 2 minutes database0 f0ed626a8152 docker-database:latest "/usr/local/bin/dock…" 2 hours ago Up 2 minutes database admin@ixre-egl-board40:~$ cat empty.json [] admin@ixre-egl-board40:~$ sudo config apply-patch empty.json -d ** DRY RUN EXECUTION ** Patch Applier: Patch application starting. Patch Applier: Patch: [] Patch Applier: Getting current config db. Patch Applier: Simulating the target full config after applying the patch. Patch Applier: Validating all JsonPatch operations are permitted on the specified fields Patch Applier: Validating target config does not have empty tables, since they do not show up in ConfigDb. Patch Applier: Sorting patch updates. Patch Applier: The patch was converted into 0 changes. Patch Applier: Applying 0 changes in order. Patch Applier: Verifying patch updates are reflected on ConfigDB. Patch Applier: Patch application completed. Patch applied successfully. admin@ixre-egl-board40:~$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 3a8ddbc204ad docker-fpm-frr:latest "/usr/bin/docker_ini…" 2 hours ago Up 2 minutes bgp0 e553479d15b1 docker-fpm-frr:latest "/usr/bin/docker_ini…" 2 hours ago Up 2 minutes bgp1 4e35cf0f962f docker-syncd-brcm-dnx:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes syncd0 f77fd9c53cd7 docker-syncd-brcm-dnx:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes syncd1 cc0377006913 docker-eventd:latest "/usr/local/bin/supe…" 2 hours ago Up 2 minutes eventd ae7c871d4e37 docker-database:latest "/usr/local/bin/dock…" 2 hours ago Up 2 minutes database1 aeb234180f94 docker-database:latest "/usr/local/bin/dock…" 2 hours ago Up 2 minutes database0 f0ed626a8152 docker-database:latest "/usr/local/bin/dock…" 2 hours ago Up 2 minutes database admin@ixre-egl-board40:~$
This has been identified by a known issue when we change a conflicting lanes for a port, but it is not related with GCU change self.