Adding support for calculating balancing in multi-lc/multi-asic case (Test_fib.py)

[vperumal]

Description of PR

Currently in multi-lc/multi-asic environment, expected count is not calculated properly and divided equally over all the interfaces, even though number of paths per asic maybe different. Added support for calculating expected count based on number of interface per asic.

The reason we needed to calculate per asic is because BGP add-path feature is currently not supported. So the current behavior for multi-lc/multi-asic case is traffic gets load balanced per asic from remote end and then load balanced based on the number of interfaces per asic. But since the script currently does a flat calculation based on number of interfaces, script fails for deviation above 25%.

For e.g.

Current behavior - say we have 4 interfaces (2 on asic0, 1 on asic1, 1 on asic2) and send 100 packets. Exp_count is 25 per port. But the traffic is divided 33 packets on asic0, 33 on asic1 and 33 asic2. Further dividing it to 16 packets on asic0 for the two interfaces, causing the testcase to fail.

New behavior - Expected count will be calculated on per_asic level so expected count for asic0 will be 33 and per interface on it to 16.

Summary: Fixes # (issue)

Type of change

• Fill x for your type of change.
• [x] Bug fix
• [ ] Testbed and Framework(new/improvement)
• [ ] Test case(new/improvement)

Back port request

• [ ] 201911
• [ ] 202012

Approach

What is the motivation for this PR?

Currently in multi-lc/multi-asic environment, expected count is not calculated properly and divided equally over all the interfaces, even though number of paths per asic maybe different.

How did you do it?

Added support for calculating expected count based on number of interface per asic.

How did you verify/test it?

Verified it on Cisco-8808 chassis

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

[vperumal]

FYI @abdosi

[gechiang]

/azp run

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[vperumal]

@abdosi , As per your request, added the balancing results:

Kindly notice that the expected count is different for the 3 asics (based on interfaces per asic in the dest port list)

14:41:36.115 root : INFO : type port(s) exp_cnt act_cnt diff(%) 14:41:36.115 root : INFO : ECMP [0, 1] 1125 1214 7.91% 14:41:36.116 root : INFO : LAG 0 607 605 -0.33% 14:41:36.116 root : INFO : LAG 1 607 609 0.33% 14:41:36.116 root : INFO : ECMP [2, 3] 1125 1118 -0.62% 14:41:36.116 root : INFO : LAG 2 559 542 -3.04% 14:41:36.116 root : INFO : LAG 3 559 576 3.04% 14:41:36.116 root : INFO : ECMP [4, 5] 1125 1120 -0.44% 14:41:36.116 root : INFO : LAG 4 560 541 -3.39% 14:41:36.116 root : INFO : LAG 5 560 579 3.39% 14:41:36.117 root : INFO : ECMP [6, 7] 1125 1066 -5.24% 14:41:36.117 root : INFO : LAG 6 533 554 3.94% 14:41:36.117 root : INFO : LAG 7 533 512 -3.94% 14:41:36.117 root : INFO : ECMP [8, 9] 1125 1119 -0.53% 14:41:36.117 root : INFO : LAG 8 559 558 -0.27% 14:41:36.118 root : INFO : LAG 9 559 561 0.27% 14:41:36.119 root : INFO : ECMP [10, 11] 1125 1113 -1.07% 14:41:36.119 root : INFO : LAG 10 556 548 -1.53% 14:41:36.119 root : INFO : LAG 11 556 565 1.53% 14:41:36.119 root : INFO : ECMP [16, 17] 658 654 -0.7% 14:41:36.119 root : INFO : LAG 16 327 305 -6.73% 14:41:36.119 root : INFO : LAG 17 327 349 6.73% 14:41:36.119 root : INFO : ECMP [13, 14] 658 675 2.49% 14:41:36.119 root : INFO : LAG 13 337 319 -5.48% 14:41:36.120 root : INFO : LAG 14 337 356 5.48% 14:41:36.120 root : INFO : ECMP [12] 658 678 2.95% 14:41:36.120 root : INFO : ECMP [15] 658 650 -1.31% 14:41:36.120 root : INFO : ECMP [18] 658 649 -1.46% 14:41:36.120 root : INFO : ECMP [19] 658 668 1.43% 14:41:36.120 root : INFO : ECMP [20] 658 633 -3.89% 14:41:36.120 root : INFO : ECMP [21] 658 624 -5.25% 14:41:36.120 root : INFO : ECMP [22] 658 693 5.22% 14:41:36.121 root : INFO : ECMP [23] 658 662 0.52% 14:41:36.121 root : INFO : ECMP [24] 833 857 2.88% 14:41:36.121 root : INFO : ECMP [25] 833 883 6.0% 14:41:36.121 root : INFO : ECMP [26] 833 825 -0.96% 14:41:36.121 root : INFO : ECMP [27] 833 822 -1.32% 14:41:36.121 root : INFO : ECMP [28] 833 834 0.12% 14:41:36.121 root : INFO : ECMP [29] 833 799 -4.08% 14:41:36.121 root : INFO : ECMP [30] 833 821 -1.44% 14:41:36.121 root : INFO : ECMP [31] 833 823 -1.2% 14:41:36.148 root : INFO : ** END TEST CASE fib_test.FibTest

[abdosi]

@arlakshm for viz.

@vperumal please add more details in PR description what trigger the need of this change. Also verify Single asic platforms are also fine after this change.

[vperumal]

@abdosi, I have updated the PR description as per your request and also verified that the change works on T0-64 and T1-64-lag profiles (Both are tested on single asic platforms)

[sanjair-git]

@vperumal, the below import for 'defaultdict' is missing in both files 'fib_test.py', 'hash_test.py' and only if we have this change, the above fix works, and the test cases are going through.

[azure-pipelines[bot]]

The pre-commit check detected issues in the files touched by this pull request. The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results: trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

ansible/roles/test/files/ptftests/fib_test.py:13:1: E265 block comment should start with '# '
ansible/roles/test/files/ptftests/fib_test.py:15:1: E265 block comment should start with '# '
ansible/roles/test/files/ptftests/fib_test.py:27:1: F401 'ptf.config' imported but unused
ansible/roles/test/files/ptftests/fib_test.py:40:1: E302 expected 2 blank lines, found 1
ansible/roles/test/files/ptftests/fib_test.py:68:5: E265 block comment should start with '# '
ansible/roles/test/files/ptftests/fib_test.py:70:5: E265 block comment should start with '# '
ansible/roles/test/files/ptftests/fib_test.py:161:24: F402 import 'fib' from line 37 shadowed by loop variable
ansible/roles/test/files/ptftests/fib_test.py:197:121: E501 line too long (138 > 120 characters)
ansible/roles/test/files/ptftests/fib_test.py:222:105: E502 the backslash is redundant between brackets
ansible/roles/test/files/ptftests/fib_test.py:223:17: E128 continuation line under-indented for visual indent
ansible/roles/test/files/ptftests/fib_test.py:237:121: E501 line too long (134 > 120 characters)
...
[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt docker container.
2. Ensure that the pre-commit package is installed:

sudo pip install pre-commit

3. Go to repository root folder
4. Install the pre-commit hooks:

pre-commit install

5. Use pre-commit to check staged file:

pre-commit

6. Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

[vperumal]

@vperumal, the below import for 'defaultdict' is missing in both files 'fib_test.py', 'hash_test.py' and only if we have this change, the above fix works, and the test cases are going through.

Thanks @sanjair-git - I have added that change

[vperumal]

@abdosi : As per your request, I have made the change applicable only for non-voq based chassis. Kindly take a look and let me know.

[azure-pipelines[bot]]

The pre-commit check detected issues in the files touched by this pull request. The detected issues may be old or new. For new issues, please try to fix them.

For old issues, it is not mandatory to fix them because they were not caused by this change. It is unfair to blame author of this pull request. But if you can take extra effort to fix the old issues as well, that would be great!

Detailed pre-commit check results: trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook

Fixing ansible/roles/test/files/ptftests/fib_test.py
Fixing ansible/roles/test/files/ptftests/py3/hash_test.py

fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

ansible/roles/test/files/ptftests/fib_test.py:13:1: E265 block comment should start with '# '
ansible/roles/test/files/ptftests/fib_test.py:15:1: E265 block comment should start with '# '
ansible/roles/test/files/ptftests/fib_test.py:27:1: F401 'ptf.config' imported but unused
ansible/roles/test/files/ptftests/fib_test.py:40:1: E302 expected 2 blank lines, found 1
...
[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt docker container.
2. Ensure that the pre-commit package is installed:

sudo pip install pre-commit

3. Go to repository root folder
4. Install the pre-commit hooks:

pre-commit install

5. Use pre-commit to check staged file:

pre-commit

6. Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

[abdosi]

@arlakshm can you please review again