sonic-swss
sonic-swss copied to clipboard
sonic-net
T2-VOQ-VS: Fix iBGP bringup issue
What I did
On T2-VOQ chassis Emulation with multi-asic linecards, iBGP sessions dont come up. Related Issue: https://github.com/sonic-net/sonic-buildimage/issues/18129
This PR fixes above issue. More details:
a. iBGP packet path is different in real target as compared to vs.
b. In real target, iBGP packets go out via IB interface, which get recycled to ingress pipeline and it does hostTable lookup for the packet to go out over Fabric link (For this to work, in Kernel Neighbor entries were programmed differently than in asic).
In kernel, neighbor entries were programmed with MyMAC and in Asic Neighbor entries were programmed with the received MAC. This was done to make sure that CPU generated packet can do hostlookup in asic (packet to get promoted to L3(my mac)).
For more details, refer to https://github.com/sonic-net/SONiC/blob/master/doc/voq/voq_hld.md#2511-routing-protocol-peering-between-sonic-instances
c. This whole packet flow is not present in VS. For vs, the CPU generated packet will go out from IB interface and go to IB Bridge and get delivered to other side.
d. Hence all the changes done in step b, need to be restored for vs (i.e. Remote Nbr entries programming need to be done against the received MAC only).
PS: Another thing to note is that on multi-asic VOQ linecards, we have same system-mac across asics. This would cause problem in establishing iBGP communication b/w same linecard asics. This will be fixed via another PR, where we will make sure, we have different system mac address across asics on the same linecard.
Why I did it This PR fixes https://github.com/sonic-net/sonic-buildimage/issues/18129.
How I verified it Verified by emulating chassis with multi-asic linecards. I could verify that iBGP sessions come up between asics of same linecard and across linecards. Details if related
Hi @prsunny , sorry the PR is not ready yet.. It added you as reviewer by default due to code ownership i guess.. Will Update the details and submit for review...
Hi @prsunny , sorry the PR is not ready yet.. It added you as reviewer by default due to code ownership i guess.. Will Update the details and submit for review...
np, i see you changed to draft PR.
can you explain how this change take care of this also sonic-net/sonic-buildimage#7434 ?
@abdosi, The earlier fix for this problem was also trying to achieve the same.. However, it was doing manipulation on Receiving LC side(to massage the mac address so that it programs with Sender LC's mac address) by assuming that All linecards will have first 5 octets of Macaddress be same and last octet will be based on switch-id..
Snippet from the earlier PR: To fix this problem, for VS platforms, if the inband interface is port type, staic neibhbors for voq neighbors are programmed with mac = neighbors's owner asic's mac. With this change, since src mac and dst mac are different and since we also program neighbors for all the remote asics, the packet will be sent out correctly.
My Current fix generalizes the approach by programming with same Mac address as sent my sender Linecards. Now, we dont have any restrictions of Mac addresses to be used across linecards (just that they need to be unique across linecards/Asic).
Hi @prsunny , could you please help review/merge this PR? Thanks!
hi @deepak-singhal0408 could you run test with 202305 image for the cherry pick?
