sonic-mgmt icon indicating copy to clipboard operation
sonic-mgmt copied to clipboard

Update test_qos_sai for ComputeAI config

Open zhixzhu opened this issue 6 months ago • 6 comments

Description of PR

Summary: Fixes # (issue) MIGSMSFT-617 [sonic-mgmt] [202311] [8111]qos.test_qos_sai failures due to ComputeAI config

Type of change

  • [ ] Bug fix
  • [ ] Testbed and Framework(new/improvement)
  • [x] Test case(new/improvement)

Back port request

  • [ ] 202012
  • [ ] 202205
  • [ ] 202305
  • [ ] 202311
  • [ ] 202405

Approach

What is the motivation for this PR?

Update test_qos_sai for ComputeAI config.

How did you do it?

Use different dscp value for lossy testing. Get dscp to queue mapping, queue, scheduler information from configuration, and use them in testcases, instead of hard-coded value.

How did you verify/test it?

Verified on 8111 with and without ComputeAI config. Test result: 8111 computeAI run

---------------------------------- generated xml file: /tmp/qos/test_qos_sai_2024-08-29-20-19-28.xml ----------------------------------
INFO:root:Can not get Allure report URL. Please check logs
------------------------------------------------------- live log sessionfinish --------------------------------------------------------
21:08:57 __init__.pytest_terminal_summary         L0067 INFO   | Can not get Allure report URL. Please check logs
======================================================= short test summary info =======================================================
PASSED qos/test_qos_sai.py::TestQosSai::testParameter[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXoffLimit[single_asic-xoff_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXoffLimit[single_asic-xoff_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXonLimit[single_asic-xon_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXonLimit[single_asic-xon_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_3]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_4]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiSharedReservationSize[single_asic-shared_res_size_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiSharedReservationSize[single_asic-shared_res_size_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiBufferPoolWatermark[single_asic-wm_buf_pool_lossless]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiBufferPoolWatermark[single_asic-wm_buf_pool_lossy]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLossyQueue[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLossyQueueVoq[single_asic-lossy_queue_voq_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDscpQueueMapping[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDwrr[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPgSharedWatermark[single_asic-wm_pg_shared_lossless]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPgSharedWatermark[single_asic-wm_pg_shared_lossy]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPGDrop[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiQSharedWatermark[single_asic-wm_q_shared_lossless]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiQSharedWatermark[single_asic-wm_q_shared_lossy]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDscpToPgMapping[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDwrrWeightChange[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiQWatermarkAllPorts[single_asic-wm_q_wm_all_ports]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLossyQueueVoqMultiSrc[single_asic]
...
======================================= 26 passed, 112 skipped, 1 warning in 2967.95s (0:49:27) =======================================
sonic@sonic-ucs-m5-8:/data/tests$ 

8111 non-computeAI run

---------------------------------- generated xml file: /tmp/qos/test_qos_sai_2024-08-29-18-29-18.xml ----------------------------------
INFO:root:Can not get Allure report URL. Please check logs
------------------------------------------------------- live log sessionfinish --------------------------------------------------------
19:19:32 __init__.pytest_terminal_summary         L0067 INFO   | Can not get Allure report URL. Please check logs
======================================================= short test summary info =======================================================
PASSED qos/test_qos_sai.py::TestQosSai::testParameter[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXoffLimit[single_asic-xoff_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXoffLimit[single_asic-xoff_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXonLimit[single_asic-xon_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXonLimit[single_asic-xon_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_3]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLosslessVoq[single_asic-lossless_voq_4]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiSharedReservationSize[single_asic-shared_res_size_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiSharedReservationSize[single_asic-shared_res_size_2]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiBufferPoolWatermark[single_asic-wm_buf_pool_lossless]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiBufferPoolWatermark[single_asic-wm_buf_pool_lossy]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLossyQueue[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLossyQueueVoq[single_asic-lossy_queue_voq_1]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDscpQueueMapping[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDwrr[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPgSharedWatermark[single_asic-wm_pg_shared_lossless]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPgSharedWatermark[single_asic-wm_pg_shared_lossy]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiPGDrop[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiQSharedWatermark[single_asic-wm_q_shared_lossless]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiQSharedWatermark[single_asic-wm_q_shared_lossy]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDscpToPgMapping[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiDwrrWeightChange[single_asic]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiQWatermarkAllPorts[single_asic-wm_q_wm_all_ports]
PASSED qos/test_qos_sai.py::TestQosSai::testQosSaiLossyQueueVoqMultiSrc[single_asic]
...
======================================= 26 passed, 112 skipped, 1 warning in 3011.80s (0:50:11) =======================================
sonic@sonic-ucs-m5-8:/data/tests$ 

Image version:

cisco@croc-aaa12-dut:~$ cat /usr/release_info 
sdk-version: 1.66.11.129
SAI_VER: 1.13.0
NPSUITE_VERSION: 1.137.1
ws-sdk-ver: HEAD-735f2e43
COMMIT_SHA: 67b8ff1e0c08bd1b0e3aab6c401978d4ea7fdbee
COMMIT_SHA_SHORT:
BRANCH: origin/master
BUILD_DATE: Thu Jul 25 16:58:54 UTC 2024
PROJECT: graphene
CUSTOMER_BUILD: 1
cisco@croc-aaa12-dut:~$ show version

SONiC Software Version: SONiC.azure_cisco_202311.14115-dirty-20240730.104536
SONiC OS Version: 11
Distribution: Debian 11.10
Kernel: 5.10.0-23-2-amd64
Build commit: 14df88942
Build date: Tue Jul 30 23:11:18 UTC 2024
Built by: sonicci@sonic-ci-17-lnx

Platform: x86_64-8111_32eh_o-r0
HwSKU: Cisco-8111-O32
ASIC: cisco-8000
ASIC Count: 1
Serial Number: FLM2641070L
Model Number: 8111-32EH-O
Hardware Revision: 1.0
Uptime: 18:37:37 up 2 days, 38 min,  1 user,  load average: 1.29, 1.20, 1.15
Date: Fri 23 Aug 2024 18:37:37

Detailed log: test_qos_sai_2024-08-29-20-19-28_compute_ai.xml.txt test_qos_sai_2024-08-29-20-19-28_compute_ai.log test_qos_sai_2024-08-29-18-29-18_non_compute_ai.xml.txt test_qos_sai_2024-08-29-18-29-18_non_compute_ai.log

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

zhixzhu avatar Aug 23 '24 19:08 zhixzhu