sonic-buildimage icon indicating copy to clipboard operation
sonic-buildimage copied to clipboard

[multi-asic] Fixed 13137 ERR python3: :- initializeGlobalConfig: SonicDBConfig Global config is already initialized

Open wumiaont opened this issue 1 year ago • 5 comments

Issue: https://github.com/sonic-net/sonic-buildimage/issues/13137 If run "show interfaces status" we often see error in syslog "SonicDBConfig Global config is already initialized".

Analysis: During investigation it is found the swsscommon.load_sonic_global_db_config() has been triggered multiple times. Especially in a multi Asic scenario.

CLI triggers "intfutil -c status" with a separate process. Within the process it will trigger IntfStatus init --> MultiAsic() init-->load_db_config()-->load_sonic_global_db_config(). This time initializeGlobalConfig() will run to the end as m_global_init flag is false.

CLI will also run get_port_config() --> db_connect_configdb() --> load_sonic_global_db_config() (for different namepaces. In our case is asic0 and asic1). This time m_global_init flag is true which causes error log and return.

Solution: There was a PR in master already fixed this issue(https://github.com/sonic-net/sonic-buildimage/pull/10960). Cherry-picked this back to 202205. Added checks in 2 other places where load_sonic_global_db_config is called without check in the code and was not covered by PR 10960.

Testing Run on chassis with 202205 loaded. Loaded the python files modified with this PR into the chassis. Run "show interface status" and will not see the error log. Show command has interface status results correctly.

wumiaont avatar Apr 09 '24 18:04 wumiaont

This is a cherry-pick fix from master. Added 2 more places with check where load_sonic_global_db_config() is called without check.

wumiaont avatar Apr 09 '24 18:04 wumiaont

@qiluo-msft Please review. Thx

wumiaont avatar Apr 12 '24 20:04 wumiaont

The code lgtm. However you should fix the master completely and backport whatever commits to 202205. Not to submit PR to 202205 directly.

qiluo-msft avatar Apr 15 '24 20:04 qiluo-msft

The code lgtm. However you should fix the master completely and backport whatever commits to 202205. Not to submit PR to 202205 directly.

OK. I will create PR against master soon.

wumiaont avatar Apr 15 '24 20:04 wumiaont

@qiluo-msft @rlhui Created PR against master https://github.com/sonic-net/sonic-buildimage/pull/18691. Please review. Unfortunately, that PR cannot be cherry-pick to 202205 as it's missing another important fix with original PR 10960 (change in portconfig.py). So best way is to let this in 202205 and have 18691 to handle fix for master.

wumiaont avatar Apr 16 '24 13:04 wumiaont

MSFT ADO: 25516271

deepak-singhal0408 avatar Apr 16 '24 23:04 deepak-singhal0408

@rlhui , could you pl help merge this change? Thanks!

deepak-singhal0408 avatar Apr 16 '24 23:04 deepak-singhal0408

@deepak-singhal0408 why is the label requested for 202305 is applied here but not from the master PR (https://github.com/sonic-net/sonic-buildimage/pull/10960) instead? This is not usually how it works...
Is the additional fix also needed for master? If so, please raise another PR in master so that the master has all the fixes and request the back port from 10960 and the new PR that you will be raising. I will remove the label "request for 202305" from this one for now.

gechiang avatar Apr 20 '24 01:04 gechiang

@deepak-singhal0408 why is the label requested for 202305 is applied here but not from the master PR (#10960) instead? This is not usually how it works... Is the additional fix also needed for master? If so, please raise another PR in master so that the master has all the fixes and request the back port from 10960 and the new PR that you will be raising. I will remove the label "request for 202305" from this one for now.

sounds good.. actually earlier there was only one PR in 202205, and we were under the impression that nothing is needed in master.. thats why I added the label here.. but later on, we made a change for this in master as well.. so yes, ideally the label should be put on master branch PR.. thanks for fixing this.

deepak-singhal0408 avatar Apr 20 '24 01:04 deepak-singhal0408