Update test_qos_sai for ComputeAI config

[zhixzhu]

Description of PR

Summary: Fixes # (issue) MIGSMSFT-617 [sonic-mgmt] [202311] [8111]qos.test_qos_sai failures due to ComputeAI config

Type of change

• [ ] Bug fix
• [ ] Testbed and Framework(new/improvement)
• [x] Test case(new/improvement)

Back port request

• [ ] 202012
• [ ] 202205
• [ ] 202305
• [ ] 202311
• [ ] 202405

Approach

What is the motivation for this PR?

Update test_qos_sai for ComputeAI config.

How did you do it?

Use different dscp value for lossy testing. Get dscp to queue mapping, queue, scheduler information from configuration, and use them in testcases, instead of hard-coded value.

How did you verify/test it?

Verified on 8111 with and without ComputeAI config. Test result: 8111 computeAI run

---------------------------------- generated xml file: /tmp/qos/test_qos_sai_2024-08-29-20-19-28.xml ----------------------------------
INFO:root:Can not get Allure report URL. Please check logs
------------------------------------------------------- live log sessionfinish --------------------------------------------------------
21:08:57 __init__.pytest_terminal_summary         L0067 INFO   | Can not get Allure report URL. Please check logs
======================================================= short test summary info =======================================================
PASSED qos/test_qos_sai.py::TestQosSai::testParameter[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXoffLimit[single_asic-xoff_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXoffLimit[single_asic-xoff_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXonLimit[single_asic-xon_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXonLimit[single_asic-xon_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_3]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_4]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiSharedReservationSize[single_asic-shared_res_size_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiSharedReservationSize[single_asic-shared_res_size_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiBufferPoolWatermark[single_asic-wm_buf_pool_lossless]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiBufferPoolWatermark[single_asic-wm_buf_pool_lossy]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLossyQueue[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLossyQueueVoq[single_asic-lossy_queue_voq_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDscpQueueMapping[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDwrr[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPgSharedWatermark[single_asic-wm_pg_shared_lossless]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPgSharedWatermark[single_asic-wm_pg_shared_lossy]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPGDrop[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiQSharedWatermark[single_asic-wm_q_shared_lossless]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiQSharedWatermark[single_asic-wm_q_shared_lossy]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDscpToPgMapping[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDwrrWeightChange[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiQWatermarkAllPorts[single_asic-wm_q_wm_all_ports]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLossyQueueVoqMultiSrc[single_asic]
...
======================================= 26 passed, 112 skipped, 1 warning in 2967.95s (0:49:27) =======================================
sonic@sonic-ucs-m5-8:/data/tests$ 

8111 non-computeAI run

---------------------------------- generated xml file: /tmp/qos/test_qos_sai_2024-08-29-18-29-18.xml ----------------------------------
INFO:root:Can not get Allure report URL. Please check logs
------------------------------------------------------- live log sessionfinish --------------------------------------------------------
19:19:32 __init__.pytest_terminal_summary         L0067 INFO   | Can not get Allure report URL. Please check logs
======================================================= short test summary info =======================================================
PASSED qos/test_qos_sai.py::TestQosSai::testParameter[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXoffLimit[single_asic-xoff_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXoffLimit[single_asic-xoff_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXonLimit[single_asic-xon_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXonLimit[single_asic-xon_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_3]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_4]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiSharedReservationSize[single_asic-shared_res_size_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiSharedReservationSize[single_asic-shared_res_size_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiBufferPoolWatermark[single_asic-wm_buf_pool_lossless]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiBufferPoolWatermark[single_asic-wm_buf_pool_lossy]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLossyQueue[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLossyQueueVoq[single_asic-lossy_queue_voq_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDscpQueueMapping[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDwrr[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPgSharedWatermark[single_asic-wm_pg_shared_lossless]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPgSharedWatermark[single_asic-wm_pg_shared_lossy]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPGDrop[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiQSharedWatermark[single_asic-wm_q_shared_lossless]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiQSharedWatermark[single_asic-wm_q_shared_lossy]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDscpToPgMapping[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDwrrWeightChange[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiQWatermarkAllPorts[single_asic-wm_q_wm_all_ports]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLossyQueueVoqMultiSrc[single_asic]
...
======================================= 26 passed, 112 skipped, 1 warning in 3011.80s (0:50:11) =======================================
sonic@sonic-ucs-m5-8:/data/tests$ 

Image version:

cisco@croc-aaa12-dut:~$ cat /usr/release_info 
sdk-version: 1.66.11.129
SAI_VER: 1.13.0
NPSUITE_VERSION: 1.137.1
ws-sdk-ver: HEAD-735f2e43
COMMIT_SHA: 67b8ff1e0c08bd1b0e3aab6c401978d4ea7fdbee
COMMIT_SHA_SHORT:
BRANCH: origin/master
BUILD_DATE: Thu Jul 25 16:58:54 UTC 2024
PROJECT: graphene
CUSTOMER_BUILD: 1
cisco@croc-aaa12-dut:~$ show version

SONiC Software Version: SONiC.azure_cisco_202311.14115-dirty-20240730.104536
SONiC OS Version: 11
Distribution: Debian 11.10
Kernel: 5.10.0-23-2-amd64
Build commit: 14df88942
Build date: Tue Jul 30 23:11:18 UTC 2024
Built by: sonicci@sonic-ci-17-lnx

Platform: x86_64-8111_32eh_o-r0
HwSKU: Cisco-8111-O32
ASIC: cisco-8000
ASIC Count: 1
Serial Number: FLM2641070L
Model Number: 8111-32EH-O
Hardware Revision: 1.0
Uptime: 18:37:37 up 2 days, 38 min,  1 user,  load average: 1.29, 1.20, 1.15
Date: Fri 23 Aug 2024 18:37:37

Detailed log: test_qos_sai_2024-08-29-20-19-28_compute_ai.xml.txt test_qos_sai_2024-08-29-20-19-28_compute_ai.log test_qos_sai_2024-08-29-18-29-18_non_compute_ai.xml.txt test_qos_sai_2024-08-29-18-29-18_non_compute_ai.log

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

[mssonicbld]

The pre-commit check detected issues in the files touched by this pull request. The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results: trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check yaml...............................................................Passed
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

tests/qos/files/cisco/qos_param_generator.py:193:121: E501 line too long (128 > 120 characters)

flake8...............................................(no files to check)Skipped
check conditional mark sort..............................................Passed

To run the pre-commit checks locally, you can follow below steps:

1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt docker container.
2. Ensure that the pre-commit package is installed:

sudo pip install pre-commit

3. Go to repository root folder
4. Install the pre-commit hooks:

pre-commit install

5. Use pre-commit to check staged file:

pre-commit

6. Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

[wsycqyz]

Re-trigger PR test.

[wsycqyz]

I am ok with this PR. @XuChen-MSFT Request your review.

[wsycqyz]

Add label for 202405 and 202311

[kevinskwang]

@zhixzhu do you have ETA for the replacement cases?

[mssonicbld]

The pre-commit check detected issues in the files touched by this pull request. The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results: trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check yaml...............................................................Passed
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

tests/qos/files/cisco/qos_param_generator.py:110:9: F841 local variable 'queue_scheduler_map' is assigned to but never used
tests/saitests/py3/sai_qos_tests.py:829:9: F841 local variable 'dual_tor_scenario' is assigned to but never used
tests/saitests/py3/sai_qos_tests.py:830:9: F841 local variable 'dual_tor' is assigned to but never used
tests/saitests/py3/sai_qos_tests.py:831:9: F841 local variable 'leaf_downstream' is assigned to but never used
tests/saitests/py3/sai_qos_tests.py:3646:33: E128 continuation line under-indented for visual indent
tests/saitests/py3/sai_qos_tests.py:3659:33: E128 continuation line under-indented for visual indent
tests/saitests/py3/sai_qos_tests.py:3664:29: E128 continuation line under-indented for visual indent
tests/saitests/py3/sai_qos_tests.py:3716:56: E231 missing whitespace after ','
tests/saitests/py3/sai_qos_tests.py:3716:73: E231 missing whitespace after ','

flake8...............................................(no files to check)Skipped
...
[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt docker container.
2. Ensure that the pre-commit package is installed:

sudo pip install pre-commit

3. Go to repository root folder
4. Install the pre-commit hooks:

pre-commit install

5. Use pre-commit to check staged file:

pre-commit

6. Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

[mssonicbld]

Cherry-pick PR to 202305: https://github.com/sonic-net/sonic-mgmt/pull/14549

[mssonicbld]

Cherry-pick PR to 202405: https://github.com/sonic-net/sonic-mgmt/pull/14651

[mssonicbld]

Cherry-pick PR to 202311: https://github.com/sonic-net/sonic-mgmt/pull/14759

[vivekverma-arista]

This has introduced a regression in qos/test_qos_sai.py::TestQosSai::testQosSaiDscpQueueMapping

06:34:12 __init__._fixture_generator_decorator    L0088 ERROR  | 
KeyError(8)
Traceback (most recent call last):
  File "/data/tests/common/plugins/log_section_start/__init__.py", line 84, in _fixture_generator_decorator
    res = next(it)
  File "/data/tests/qos/qos_sai_base.py", line 2455, in tc_to_dscp_count
    for dscp, tc in dscp_to_tc_map.items():
KeyError: 8

[zhixzhu]

@vivekverma-arista PR https://github.com/sonic-net/sonic-mgmt/pull/14715 fixed this error.

[vivekverma-arista]

@vivekverma-arista PR #14715 fixed this error.

This is a different error for which I have raised #15109 to fix it.