sonic-swss [pfc] Introduce pfc config bitmask to track user-configured pfc enabled TCs on physical port

What I did Introduce PFC config bitmask to track user-configured PFC enabled TCs on physical port

This allows separate tracking of user-configured PFC bitmask (pfc_enable field in CONFIG_DB) from PFC bitmask status in ASIC (i.e., pfcwd config). As when pfc wd is enabled and PFC storm occurs on a lossless TC, PFC bit on that TC is cleared in ASIC during the storm by PfcWdLossyHandler, making PFC bitmask status in ASIC to be different from that configured by user.

Such a difference causes the following misleading log message in syslog when a lossless TC is in PFC storm and pfc wd drop action is in place:

Aug 20 10:28:43.590517 sonic WARNING swss#orchagent: :- pfcFrameCounterCheck: Got PFC 933416836 frame(s) on lossy queue 3 port Ethernet0

Why I did it Fix https://github.com/Azure/sonic-buildimage/issues/5910

How I verified it On brcm

Details if related Depends on

[x] https://github.com/Azure/sonic-sairedis/pull/814

When user wants to change PFC enable bits on TCs, if pfc wd is enabled, user still needs to toggle pfc wd stop and start to ensure pfc wd state machine is stopped on a PFC-disabled TC and is running on a PFC-enabled TC (https://github.com/Azure/sonic-swss/issues/842). However, before this change, command sequence matters: pfcwd must be stopped before PFC bits change on TCs, followed by pfcwd start. With this change, restriction on command sequence is no longer needed, i.e., PFC bits change on TC can be issued either before or after pfcwd stop, followed by pfcwd start.

Removing manual intervention described above (pfc wd stop and start toggle in changing PFC bits on TCs) is out of the scope of this PR, and will be in follow-up ones.

Covers fix to allow disabling last lossless TC(s) in pending PR https://github.com/Azure/sonic-swss/pull/1055

vs tests

test case 1: Clear PFC enable bits on a physical port

This is to verify fix to https://github.com/Azure/sonic-swss/pull/1055

Set PFC enable on {port, TC}
Clear PFC enable on {port, TC}, and verify port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL is set 0

Before the change, test fails: port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL remains 24

================================================================================ FAILURES =================================================================================
_____________________________________________________________________ TestPfc.test_pfc_en_bits_clear ______________________________________________________________________

self = <test_pfc.TestPfc object at 0x7f254af3d0f0>, dvs = <conftest.DockerVirtualSwitch object at 0x7f254af3d9e8>, testlog = <function testlog at 0x7f254af590d0>

    def test_pfc_en_bits_clear(self, dvs, testlog):
        PORT_UNDER_TEST = "Ethernet64"
    
        pfc_tcs = [3, 4]
        setPortPfc(dvs, PORT_UNDER_TEST, pfc_tcs)
    
        # Verify pfc enable bits in ASIC_DB
        port_oid = dvs.asicdb.portnamemap[PORT_UNDER_TEST]
        pfc = getPortAttr(dvs, port_oid, 'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL')
        assert pfc == getBitMaskStr(pfc_tcs)
    
        pfc_tcs = []
        setPortPfc(dvs, PORT_UNDER_TEST, pfc_tcs)
    
        # Verify pfc enable bits all cleared in ASIC_DB
        pfc = getPortAttr(dvs, port_oid, 'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL')
>       assert pfc == getBitMaskStr(pfc_tcs)
E       AssertionError: assert '24' == '0'
E         - 0
E         + 24

test_pfc.py:120: AssertionError
========================================================================= short test summary info =========================================================================
FAILED test_pfc.py::TestPfc::test_pfc_en_bits_clear - AssertionError: assert '24' == '0'
====================================================================== 1 failed in 61.19s (0:01:01) =======================================================================

test case 2: Separate user and wd PFC enable bits on a physical port

Test is designed around the aforementioned step restriction on changing PFC enable bits on TCs, if pfc wd is enabled---Without this change, pfcwd must be stopped before PFC bits change on TCs, followed by pfcwd start.

Set PFC enable on {port, TC 3}. Port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL 8'b00001000 == 8
Set PFC WD config on port to start PFC WD state machine on {port, TC 3}
Mimic PFC storm on {port queue 3} using DEBUG_STORM. Port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL changes to 8'b00000000 == 0
Re-set PFC enable on {port, TC 3}. Port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL remains 8'b00000000 == 0. W/o this change, Port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL changes to 8'b00001000 == 8. Test fail point 1
Change PFC enable from {port, TC 3} to {port, TC 4}. Port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL changes to 8'b00010000 == 16
Remove PFC WD config on port. Port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL remains 8'b00010000 == 16. W/o this change, port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL changes to 8'b00011000 == 24. Test fail point 2

Test fail point 1

================================================================================ FAILURES =================================================================================
_______________________________________________________________ TestPfcWd.test_pfc_en_bits_user_wd_cfg_sep ________________________________________________________________

self = <test_pfcwd.TestPfcWd object at 0x7f173ee80c88>, dvs = <conftest.DockerVirtualSwitch object at 0x7f173ef08d68>, testlog = <function testlog at 0x7f173ef520d0>

    def test_pfc_en_bits_user_wd_cfg_sep(self, dvs, testlog):
        self.connect_dbs(dvs)
    
        # Enable pfc wd flex counter polling
        self.enable_flex_counter(CFG_FLEX_COUNTER_TABLE_PFCWD_KEY)
        # Verify pfc wd flex counter status published to FLEX_COUNTER_DB FLEX_COUNTER_GROUP_TABLE by flex counter orch
        fv_dict = {
            FLEX_COUNTER_STATUS: ENABLE,
        }
        self.check_db_fvs(self.flex_cntr_db, FC_FLEX_COUNTER_GROUP_TABLE_NAME, FC_FLEX_COUNTER_GROUP_TABLE_PFC_WD_KEY, fv_dict)
    
        # Enable pfc on tc 3
        pfc_tcs = [QUEUE_3]
        self.set_port_pfc(PORT_UNDER_TEST, pfc_tcs)
    
        # Verify pfc enable bits in ASIC_DB
        port_oid = dvs.asicdb.portnamemap[PORT_UNDER_TEST]
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "8",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Start pfc wd (config) on port
        self.start_port_pfcwd(PORT_UNDER_TEST)
        # Verify port level counter to poll published to FLEX_COUNTER_DB FLEX_COUNTER_TABLE by pfc wd orch
        self.check_db_key_existence(self.flex_cntr_db, FC_FLEX_COUNTER_TABLE_NAME,
                                    "{}:{}".format(FC_FLEX_COUNTER_TABLE_PFC_WD_KEY_PREFIX, port_oid))
        # Verify queue level counter to poll published to FLEX_COUNTER_DB FLEX_COUNTER_TABLE by pfc wd orch
        queue_oid = self.get_queue_oid(dvs, PORT_UNDER_TEST, QUEUE_3)
        self.check_db_key_existence(self.flex_cntr_db, FC_FLEX_COUNTER_TABLE_NAME,
                                    "{}:{}".format(FC_FLEX_COUNTER_TABLE_PFC_WD_KEY_PREFIX, queue_oid))
    
        # Verify pfc enable bits stay unchanged in ASIC_DB
        time.sleep(2)
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "8",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Start pfc storm on queue 3
        self.start_queue_pfc_storm(queue_oid)
        # Verify queue in storm from COUNTERS_DB
        fv_dict = {
            PFC_WD_STATUS: STORMED,
        }
        self.check_db_fvs(self.cntrs_db, CNTR_COUNTERS_TABLE_NAME, queue_oid, fv_dict)
    
        # Verify pfc enable bits change in ASIC_DB
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "0",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Re-set pfc enable on tc 3
        pfc_tcs = [QUEUE_3]
        self.set_port_pfc(PORT_UNDER_TEST, pfc_tcs)
    
        # Verify pfc enable bits stay unchanged in ASIC_DB
        time.sleep(2)
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "0",
        }
>       self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)

test_pfcwd.py:246: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_pfcwd.py:165: in check_db_fvs
    db.wait_for_field_match(table_name, key, fv_dict)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <dvslib.dvs_database.DVSDatabase object at 0x7f173eeeffd0>, table_name = 'ASIC_STATE:SAI_OBJECT_TYPE_PORT', key = 'oid:0x1000000000020'
expected_fields = {'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL': '0'}, polling_config = PollingConfig(polling_interval=0.01, timeout=5.0, strict=True), failure_message = None

    def wait_for_field_match(
        self,
        table_name: str,
        key: str,
        expected_fields: Dict[str, str],
        polling_config: PollingConfig = PollingConfig(),
        failure_message: str = None,
    ) -> Dict[str, str]:
        """Wait for the entry stored at `key` to have the specified field/values and retrieve it.
    
        This method is useful if you only care about the contents of a subset of the fields stored
        in the specified entry.
    
        Args:
            table_name: The name of the table where the entry is stored.
            key: The key that maps to the entry being checked.
            expected_fields: The fields and their values we expect to see in the entry.
            polling_config: The parameters to use to poll the db.
            failure_message: The message to print if the call times out. This will only take effect
                if the PollingConfig is set to strict.
    
        Returns:
            The entry stored at `key`. If no entry is found, then an empty Dict is returned.
        """
    
        def access_function():
            fv_pairs = self.get_entry(table_name, key)
            return (
                all(fv_pairs.get(k) == v for k, v in expected_fields.items()),
                fv_pairs,
            )
    
        status, result = wait_for_result(
            access_function, self._disable_strict_polling(polling_config)
        )
    
        if not status:
            message = failure_message or (
                f"Expected field/value pairs not found: expected={expected_fields}, "
                f'received={result}, key="{key}", table="{table_name}"'
            )
>           assert not polling_config.strict, message
E           AssertionError: Expected field/value pairs not found: expected={'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL': '0'}, received={'NULL': 'NULL', 'SAI_PORT_ATTR_ADMIN_STATE': 'false', 'SAI_PORT_ATTR_SPEED': '100000', 'SAI_PORT_ATTR_MTU': '9122', 'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL': '8', 'SAI_PORT_ATTR_INGRESS_ACL': 'oid:0xb0000000005f4', 'SAI_PORT_ATTR_EGRESS_ACL': 'oid:0xb0000000005f8'}, key="oid:0x1000000000020", table="ASIC_STATE:SAI_OBJECT_TYPE_PORT"

dvslib/dvs_database.py:203: AssertionError
========================================================================= short test summary info =========================================================================
FAILED test_pfcwd.py::TestPfcWd::test_pfc_en_bits_user_wd_cfg_sep - AssertionError: Expected field/value pairs not found: expected={'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL...
====================================================================== 1 failed in 69.25s (0:01:09) =======================================================================

Test fail point 2

================================================================================ FAILURES =================================================================================
_______________________________________________________________ TestPfcWd.test_pfc_en_bits_user_wd_cfg_sep ________________________________________________________________

self = <test_pfcwd.TestPfcWd object at 0x7ffafdd3e240>, dvs = <conftest.DockerVirtualSwitch object at 0x7ffafdd3e208>, testlog = <function testlog at 0x7ffafde100d0>

    def test_pfc_en_bits_user_wd_cfg_sep(self, dvs, testlog):
        self.connect_dbs(dvs)
    
        # Enable pfc wd flex counter polling
        self.enable_flex_counter(CFG_FLEX_COUNTER_TABLE_PFCWD_KEY)
        # Verify pfc wd flex counter status published to FLEX_COUNTER_DB FLEX_COUNTER_GROUP_TABLE by flex counter orch
        fv_dict = {
            FLEX_COUNTER_STATUS: ENABLE,
        }
        self.check_db_fvs(self.flex_cntr_db, FC_FLEX_COUNTER_GROUP_TABLE_NAME, FC_FLEX_COUNTER_GROUP_TABLE_PFC_WD_KEY, fv_dict)
    
        # Enable pfc on tc 3
        pfc_tcs = [QUEUE_3]
        self.set_port_pfc(PORT_UNDER_TEST, pfc_tcs)
    
        # Verify pfc enable bits in ASIC_DB
        port_oid = dvs.asicdb.portnamemap[PORT_UNDER_TEST]
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "8",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Start pfc wd (config) on port
        self.start_port_pfcwd(PORT_UNDER_TEST)
        # Verify port level counter to poll published to FLEX_COUNTER_DB FLEX_COUNTER_TABLE by pfc wd orch
        self.check_db_key_existence(self.flex_cntr_db, FC_FLEX_COUNTER_TABLE_NAME,
                                    "{}:{}".format(FC_FLEX_COUNTER_TABLE_PFC_WD_KEY_PREFIX, port_oid))
        # Verify queue level counter to poll published to FLEX_COUNTER_DB FLEX_COUNTER_TABLE by pfc wd orch
        queue_oid = self.get_queue_oid(dvs, PORT_UNDER_TEST, QUEUE_3)
        self.check_db_key_existence(self.flex_cntr_db, FC_FLEX_COUNTER_TABLE_NAME,
                                    "{}:{}".format(FC_FLEX_COUNTER_TABLE_PFC_WD_KEY_PREFIX, queue_oid))
    
        # Verify pfc enable bits stay unchanged in ASIC_DB
        time.sleep(2)
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "8",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Start pfc storm on queue 3
        self.start_queue_pfc_storm(queue_oid)
        # Verify queue in storm from COUNTERS_DB
        fv_dict = {
            PFC_WD_STATUS: STORMED,
        }
        self.check_db_fvs(self.cntrs_db, CNTR_COUNTERS_TABLE_NAME, queue_oid, fv_dict)
    
        # Verify pfc enable bits change in ASIC_DB
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "0",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Re-set pfc enable on tc 3
        pfc_tcs = [QUEUE_3]
        self.set_port_pfc(PORT_UNDER_TEST, pfc_tcs)
    
        # Verify pfc enable bits stay unchanged in ASIC_DB
        #time.sleep(2)
        #fv_dict = {
        #    "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "0",
        #}
        #self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Change pfc enable bits: disable pfc on tc 3, and enable pfc on tc 4
        pfc_tcs = [QUEUE_4]
        self.set_port_pfc(PORT_UNDER_TEST, pfc_tcs)
    
        # Verify pfc enable bits change in ASIC_DB
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "16",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Stop pfc wd on port (i.e., remove pfc wd config from port)
        self.stop_port_pfcwd(PORT_UNDER_TEST)
        # Verify port level counter removed from FLEX_COUNTER_DB
        self.check_db_key_removal(self.flex_cntr_db, FC_FLEX_COUNTER_TABLE_NAME,
                                  "{}:{}".format(FC_FLEX_COUNTER_TABLE_PFC_WD_KEY_PREFIX, port_oid))
        # Verify queue level counter removed from FLEX_COUNTER_DB
        queue_oid = self.get_queue_oid(dvs, PORT_UNDER_TEST, QUEUE_3)
        self.check_db_key_removal(self.flex_cntr_db, FC_FLEX_COUNTER_TABLE_NAME,
                                  "{}:{}".format(FC_FLEX_COUNTER_TABLE_PFC_WD_KEY_PREFIX, queue_oid))
    
        # Verify pfc wd fields removed from COUNTERS_DB
        fields = [PFC_WD_STATUS]
        self.check_db_fields_removal(self.cntrs_db, CNTR_COUNTERS_TABLE_NAME, queue_oid, fields)
    
        # Verify pfc enable bits in ASIC_DB (stay unchanged)
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "16",
        }
>       self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)

test_pfcwd.py:276: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_pfcwd.py:165: in check_db_fvs
    db.wait_for_field_match(table_name, key, fv_dict)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <dvslib.dvs_database.DVSDatabase object at 0x7ffafdd0af60>, table_name = 'ASIC_STATE:SAI_OBJECT_TYPE_PORT', key = 'oid:0x1000000000020'
expected_fields = {'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL': '16'}, polling_config = PollingConfig(polling_interval=0.01, timeout=5.0, strict=True), failure_message = None

    def wait_for_field_match(
        self,
        table_name: str,
        key: str,
        expected_fields: Dict[str, str],
        polling_config: PollingConfig = PollingConfig(),
        failure_message: str = None,
    ) -> Dict[str, str]:
        """Wait for the entry stored at `key` to have the specified field/values and retrieve it.
    
        This method is useful if you only care about the contents of a subset of the fields stored
        in the specified entry.
    
        Args:
            table_name: The name of the table where the entry is stored.
            key: The key that maps to the entry being checked.
            expected_fields: The fields and their values we expect to see in the entry.
            polling_config: The parameters to use to poll the db.
            failure_message: The message to print if the call times out. This will only take effect
                if the PollingConfig is set to strict.
    
        Returns:
            The entry stored at `key`. If no entry is found, then an empty Dict is returned.
        """
    
        def access_function():
            fv_pairs = self.get_entry(table_name, key)
            return (
                all(fv_pairs.get(k) == v for k, v in expected_fields.items()),
                fv_pairs,
            )
    
        status, result = wait_for_result(
            access_function, self._disable_strict_polling(polling_config)
        )
    
        if not status:
            message = failure_message or (
                f"Expected field/value pairs not found: expected={expected_fields}, "
                f'received={result}, key="{key}", table="{table_name}"'
            )
>           assert not polling_config.strict, message
E           AssertionError: Expected field/value pairs not found: expected={'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL': '16'}, received={'NULL': 'NULL', 'SAI_PORT_ATTR_ADMIN_STATE': 'false', 'SAI_PORT_ATTR_SPEED': '100000', 'SAI_PORT_ATTR_MTU': '9122', 'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL': '24', 'SAI_PORT_ATTR_INGRESS_ACL': 'oid:0xb0000000005f4', 'SAI_PORT_ATTR_EGRESS_ACL': 'oid:0x0'}, key="oid:0x1000000000020", table="ASIC_STATE:SAI_OBJECT_TYPE_PORT"

dvslib/dvs_database.py:203: AssertionError
========================================================================= short test summary info =========================================================================
FAILED test_pfcwd.py::TestPfcWd::test_pfc_en_bits_user_wd_cfg_sep - AssertionError: Expected field/value pairs not found: expected={'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL...
====================================================================== 1 failed in 67.40s (0:01:07) =======================================================================

Variation: install PFC WD drop action using big red switch mode in place of faking PFC storm.

Set PFC enable on {port, TC 4}. Port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL 8'b00010000 == 16
Set PFC WD config on port to start PFC WD state machine on {port, TC 4}
Enable big red switch mode to install PFC WD drop action on port. Port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL changes to 8'b00000000 == 0
Re-set PFC enable on {port, TC 4}. Port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL remains 8'b00000000 == 0. W/o this change, Port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL changes to 8'b00010000 == 16. Test fail point 1
Change PFC enable from {port, TC 4} to {port, TC 3}. Port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL changes to 8'b00001000 == 8
Disable big red switch mode to remove PFC WD drop action on port. Port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL remains 8'b00001000 == 8. W/o this change, port SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL changes to 8'b00011000 == 24. Test fail point 2

Test fail point 1

========================================================================== FAILURES ==========================================================================
_______________________________________________________ TestPfcWd.test_pfc_en_bits_user_wd_cfg_sep_brs _______________________________________________________

self = <test_pfcwd.TestPfcWd object at 0x7f38b25165c0>, dvs = <conftest.DockerVirtualSwitch object at 0x7f38b252a518>
testlog = <function testlog at 0x7f38b25450d0>

    def test_pfc_en_bits_user_wd_cfg_sep_brs(self, dvs, testlog):
        self.connect_dbs(dvs)
    
        # Enable pfc wd flex counter polling
        self.enable_flex_counter(CFG_FLEX_COUNTER_TABLE_PFCWD_KEY)
        # Verify pfc wd flex counter status published to FLEX_COUNTER_DB FLEX_COUNTER_GROUP_TABLE by flex counter orch
        fv_dict = {
            FLEX_COUNTER_STATUS: ENABLE,
        }
        self.check_db_fvs(self.flex_cntr_db, FC_FLEX_COUNTER_GROUP_TABLE_NAME, FC_FLEX_COUNTER_GROUP_TABLE_PFC_WD_KEY, fv_dict)
    
        # Enable pfc on tc 4
        pfc_tcs = [QUEUE_4]
        self.set_port_pfc(PORT_UNDER_TEST, pfc_tcs)
    
        # Verify pfc enable bits in ASIC_DB
        port_oid = dvs.asicdb.portnamemap[PORT_UNDER_TEST]
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "16",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Start pfc wd (config) on port
        self.start_port_pfcwd(PORT_UNDER_TEST)
        # Verify port level counter to poll published to FLEX_COUNTER_DB FLEX_COUNTER_TABLE by pfc wd orch
        self.check_db_key_existence(self.flex_cntr_db, FC_FLEX_COUNTER_TABLE_NAME,
                                    "{}:{}".format(FC_FLEX_COUNTER_TABLE_PFC_WD_KEY_PREFIX, port_oid))
        # Verify queue level counter to poll published to FLEX_COUNTER_DB FLEX_COUNTER_TABLE by pfc wd orch
        queue_oid = self.get_queue_oid(dvs, PORT_UNDER_TEST, QUEUE_4)
        self.check_db_key_existence(self.flex_cntr_db, FC_FLEX_COUNTER_TABLE_NAME,
                                    "{}:{}".format(FC_FLEX_COUNTER_TABLE_PFC_WD_KEY_PREFIX, queue_oid))
    
        # Verify pfc enable bits stay unchanged in ASIC_DB
        time.sleep(2)
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "16",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Enable big red switch
        self.enable_big_red_switch()
        # Verify queue 4 in brs from COUNTERS_DB
        fv_dict = {
            BIG_RED_SWITCH_MODE: ENABLE,
        }
        self.check_db_fvs(self.cntrs_db, CNTR_COUNTERS_TABLE_NAME, queue_oid, fv_dict)
    
        # Verify pfc enable bits change in ASIC_DB
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "0",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Re-set pfc enable on tc 4
        pfc_tcs = [QUEUE_4]
        self.set_port_pfc(PORT_UNDER_TEST, pfc_tcs)
    
        # Verify pfc enable bits stay unchanged in ASIC_DB
        time.sleep(2)
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "0",
        }
>       self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)

test_pfcwd.py:362: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_pfcwd.py:175: in check_db_fvs
    db.wait_for_field_match(table_name, key, fv_dict)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dvslib.dvs_database.DVSDatabase object at 0x7f38b2479c18>, table_name = 'ASIC_STATE:SAI_OBJECT_TYPE_PORT', key = 'oid:0x1000000000020'
expected_fields = {'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL': '0'}, polling_config = PollingConfig(polling_interval=0.01, timeout=5.0, strict=True)
failure_message = None

    def wait_for_field_match(
        self,
        table_name: str,
        key: str,
        expected_fields: Dict[str, str],
        polling_config: PollingConfig = PollingConfig(),
        failure_message: str = None,
    ) -> Dict[str, str]:
        """Wait for the entry stored at `key` to have the specified field/values and retrieve it.
    
        This method is useful if you only care about the contents of a subset of the fields stored
        in the specified entry.
    
        Args:
            table_name: The name of the table where the entry is stored.
            key: The key that maps to the entry being checked.
            expected_fields: The fields and their values we expect to see in the entry.
            polling_config: The parameters to use to poll the db.
            failure_message: The message to print if the call times out. This will only take effect
                if the PollingConfig is set to strict.
    
        Returns:
            The entry stored at `key`. If no entry is found, then an empty Dict is returned.
        """
    
        def access_function():
            fv_pairs = self.get_entry(table_name, key)
            return (
                all(fv_pairs.get(k) == v for k, v in expected_fields.items()),
                fv_pairs,
            )
    
        status, result = wait_for_result(
            access_function, self._disable_strict_polling(polling_config)
        )
    
        if not status:
            message = failure_message or (
                f"Expected field/value pairs not found: expected={expected_fields}, "
                f'received={result}, key="{key}", table="{table_name}"'
            )
>           assert not polling_config.strict, message
E           AssertionError: Expected field/value pairs not found: expected={'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL': '0'}, received={'NULL': 'NULL', 'SAI_PORT_ATTR_ADMIN_STATE': 'false', 'SAI_PORT_ATTR_SPEED': '100000', 'SAI_PORT_ATTR_MTU': '9122', 'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL': '16', 'SAI_PORT_ATTR_INGRESS_ACL': 'oid:0xb0000000005f4', 'SAI_PORT_ATTR_EGRESS_ACL': 'oid:0xb0000000005f8'}, key="oid:0x1000000000020", table="ASIC_STATE:SAI_OBJECT_TYPE_PORT"

dvslib/dvs_database.py:203: AssertionError
================================================================== short test summary info ===================================================================
FAILED test_pfcwd.py::TestPfcWd::test_pfc_en_bits_user_wd_cfg_sep_brs - AssertionError: Expected field/value pairs not found: expected={'SAI_PORT_ATTR_PRIO...
================================================================ 1 failed in 68.68s (0:01:08) ================================================================

Test fail point 2

========================================================================== FAILURES ==========================================================================
_______________________________________________________ TestPfcWd.test_pfc_en_bits_user_wd_cfg_sep_brs _______________________________________________________

self = <test_pfcwd.TestPfcWd object at 0x7f0991c37400>, dvs = <conftest.DockerVirtualSwitch object at 0x7f0991c37160>
testlog = <function testlog at 0x7f0991d440d0>

    def test_pfc_en_bits_user_wd_cfg_sep_brs(self, dvs, testlog):
        self.connect_dbs(dvs)
    
        # Enable pfc wd flex counter polling
        self.enable_flex_counter(CFG_FLEX_COUNTER_TABLE_PFCWD_KEY)
        # Verify pfc wd flex counter status published to FLEX_COUNTER_DB FLEX_COUNTER_GROUP_TABLE by flex counter orch
        fv_dict = {
            FLEX_COUNTER_STATUS: ENABLE,
        }
        self.check_db_fvs(self.flex_cntr_db, FC_FLEX_COUNTER_GROUP_TABLE_NAME, FC_FLEX_COUNTER_GROUP_TABLE_PFC_WD_KEY, fv_dict)
    
        # Enable pfc on tc 4
        pfc_tcs = [QUEUE_4]
        self.set_port_pfc(PORT_UNDER_TEST, pfc_tcs)
    
        # Verify pfc enable bits in ASIC_DB
        port_oid = dvs.asicdb.portnamemap[PORT_UNDER_TEST]
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "16",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Start pfc wd (config) on port
        self.start_port_pfcwd(PORT_UNDER_TEST)
        # Verify port level counter to poll published to FLEX_COUNTER_DB FLEX_COUNTER_TABLE by pfc wd orch
        self.check_db_key_existence(self.flex_cntr_db, FC_FLEX_COUNTER_TABLE_NAME,
                                    "{}:{}".format(FC_FLEX_COUNTER_TABLE_PFC_WD_KEY_PREFIX, port_oid))
        # Verify queue level counter to poll published to FLEX_COUNTER_DB FLEX_COUNTER_TABLE by pfc wd orch
        queue_oid = self.get_queue_oid(dvs, PORT_UNDER_TEST, QUEUE_4)
        self.check_db_key_existence(self.flex_cntr_db, FC_FLEX_COUNTER_TABLE_NAME,
                                    "{}:{}".format(FC_FLEX_COUNTER_TABLE_PFC_WD_KEY_PREFIX, queue_oid))
    
        # Verify pfc enable bits stay unchanged in ASIC_DB
        time.sleep(2)
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "16",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Enable big red switch
        self.enable_big_red_switch()
        # Verify queue 4 in brs from COUNTERS_DB
        fv_dict = {
            BIG_RED_SWITCH_MODE: ENABLE,
        }
        self.check_db_fvs(self.cntrs_db, CNTR_COUNTERS_TABLE_NAME, queue_oid, fv_dict)
    
        # Verify pfc enable bits change in ASIC_DB
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "0",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Re-set pfc enable on tc 4
        pfc_tcs = [QUEUE_4]
        self.set_port_pfc(PORT_UNDER_TEST, pfc_tcs)
    
        # Verify pfc enable bits stay unchanged in ASIC_DB
        #time.sleep(2)
        #fv_dict = {
        #    "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "0",
        #}
        #self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Change pfc enable bits: disable pfc on tc 4, and enable pfc on tc 3
        pfc_tcs = [QUEUE_3]
        self.set_port_pfc(PORT_UNDER_TEST, pfc_tcs)
    
        # Verify pfc enable bits change in ASIC_DB
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "8",
        }
        self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)
    
        # Disable big red switch
        self.disable_big_red_switch()
        # Verify brs field removed from COUNTERS_DB
        fields = [BIG_RED_SWITCH_MODE]
        self.check_db_fields_removal(self.cntrs_db, CNTR_COUNTERS_TABLE_NAME, queue_oid, fields)
    
        # Verify pfc enable bits in ASIC_DB (stay unchanged)
        fv_dict = {
            "SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL": "8",
        }
>       self.check_db_fvs(self.asic_db, ASIC_PORT_TABLE_NAME, port_oid, fv_dict)

test_pfcwd.py:384: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_pfcwd.py:175: in check_db_fvs
    db.wait_for_field_match(table_name, key, fv_dict)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dvslib.dvs_database.DVSDatabase object at 0x7f0991c80d30>, table_name = 'ASIC_STATE:SAI_OBJECT_TYPE_PORT', key = 'oid:0x1000000000020'
expected_fields = {'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL': '8'}, polling_config = PollingConfig(polling_interval=0.01, timeout=5.0, strict=True)
failure_message = None

    def wait_for_field_match(
        self,
        table_name: str,
        key: str,
        expected_fields: Dict[str, str],
        polling_config: PollingConfig = PollingConfig(),
        failure_message: str = None,
    ) -> Dict[str, str]:
        """Wait for the entry stored at `key` to have the specified field/values and retrieve it.
    
        This method is useful if you only care about the contents of a subset of the fields stored
        in the specified entry.
    
        Args:
            table_name: The name of the table where the entry is stored.
            key: The key that maps to the entry being checked.
            expected_fields: The fields and their values we expect to see in the entry.
            polling_config: The parameters to use to poll the db.
            failure_message: The message to print if the call times out. This will only take effect
                if the PollingConfig is set to strict.
    
        Returns:
            The entry stored at `key`. If no entry is found, then an empty Dict is returned.
        """
    
        def access_function():
            fv_pairs = self.get_entry(table_name, key)
            return (
                all(fv_pairs.get(k) == v for k, v in expected_fields.items()),
                fv_pairs,
            )
    
        status, result = wait_for_result(
            access_function, self._disable_strict_polling(polling_config)
        )
    
        if not status:
            message = failure_message or (
                f"Expected field/value pairs not found: expected={expected_fields}, "
                f'received={result}, key="{key}", table="{table_name}"'
            )
>           assert not polling_config.strict, message
E           AssertionError: Expected field/value pairs not found: expected={'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL': '8'}, received={'NULL': 'NULL', 'SAI_PORT_ATTR_ADMIN_STATE': 'false', 'SAI_PORT_ATTR_SPEED': '100000', 'SAI_PORT_ATTR_MTU': '9122', 'SAI_PORT_ATTR_PRIORITY_FLOW_CONTROL': '24', 'SAI_PORT_ATTR_INGRESS_ACL': 'oid:0xb0000000005f4', 'SAI_PORT_ATTR_EGRESS_ACL': 'oid:0x0'}, key="oid:0x1000000000020", table="ASIC_STATE:SAI_OBJECT_TYPE_PORT"

dvslib/dvs_database.py:203: AssertionError
================================================================== short test summary info ===================================================================
FAILED test_pfcwd.py::TestPfcWd::test_pfc_en_bits_user_wd_cfg_sep_brs - AssertionError: Expected field/value pairs not found: expected={'SAI_PORT_ATTR_PRIO...
================================================================ 1 failed in 66.73s (0:01:06) ================================================================

Jan 26 '21 05:01 wendani

/Azurepipelines run

Feb 05 '21 01:02 wendani

Commenter does not have sufficient privileges for PR 1612 in repo Azure/sonic-swss

Feb 05 '21 01:02 azure-pipelines[bot]

can you add a test case for this?

Mar 05 '21 14:03 lguohan

others look good to me.

Mar 05 '21 14:03 lguohan

@wendani , FYI, https://github.com/Azure/sonic-swss/pull/1360 changes the signature of setPortPfc and getPortPfc methods

Mar 11 '21 04:03 neethajohn

This pull request fixes 2 alerts when merging 47a8fad7420509120b98d831aecd26905d686ac2 into 89d0728401efc53c833c8fc1cc82099ac86165f1 - view on LGTM.com

fixed alerts:

2 for Unused import

Mar 28 '21 07:03 lgtm-com[bot]

This pull request fixes 2 alerts when merging f6d6e6d75891c69fadcd3fcf917d961d75075f01 into 89d0728401efc53c833c8fc1cc82099ac86165f1 - view on LGTM.com

fixed alerts:

2 for Unused import

Mar 28 '21 23:03 lgtm-com[bot]

This pull request fixes 2 alerts when merging e1bd1254cc729360bab9ca0e328a22fb00f04a3e into 15818ad5748eb836d8b75d7282cef9aade156f3f - view on LGTM.com

fixed alerts:

2 for Unused import

Mar 30 '21 03:03 lgtm-com[bot]

This pull request fixes 2 alerts when merging 92a0e35d85b41679864a88d1d3632d50736a70a8 into 15818ad5748eb836d8b75d7282cef9aade156f3f - view on LGTM.com

fixed alerts:

2 for Unused import

Mar 30 '21 07:03 lgtm-com[bot]

This pull request fixes 2 alerts when merging cbb2410342bb84a480f1c65c318cfaeb9f9fc662 into 691bd3000f10d167233bac3b37c7ce4e4c478072 - view on LGTM.com

fixed alerts:

2 for Unused import

Mar 31 '21 21:03 lgtm-com[bot]

This pull request fixes 2 alerts when merging 6673a9c3104aebf16df5d491c0618ec454bb2c59 into cba65760393d2380785c407158609f4782777718 - view on LGTM.com

fixed alerts:

2 for Unused import

Apr 02 '21 07:04 lgtm-com[bot]

This pull request fixes 2 alerts when merging c491f08b4f8c98550db6534bb94d745722bb7d8c into 872b5cb9a2a398a086f4646fe134c199919b6c92 - view on LGTM.com

fixed alerts:

2 for Unused import

Apr 03 '21 22:04 lgtm-com[bot]

can you add a test case for this?

Done

Apr 04 '21 00:04 wendani

@wendani , FYI, #1360 changes the signature of setPortPfc and getPortPfc methods

What is your plan to move forward?

Apr 04 '21 00:04 wendani

This pull request fixes 2 alerts when merging 8248910b22173b038dbac4e71ac8566362ca3d0b into 872b5cb9a2a398a086f4646fe134c199919b6c92 - view on LGTM.com

fixed alerts:

2 for Unused import

Apr 05 '21 11:04 lgtm-com[bot]

@wendani , FYI, #1360 changes the signature of setPortPfc and getPortPfc methods

What is your plan to move forward?

That PR needs some rework. so you should be good. will review these changes soon

Apr 05 '21 17:04 neethajohn

This pull request fixes 2 alerts when merging 1d4f9bcfe85d62e204d7cab17cf9c4c8c598a486 into 8bc7aee23b53bd2ca577695d9f09a44d4189f4b9 - view on LGTM.com

fixed alerts:

2 for Unused import

Apr 17 '21 02:04 lgtm-com[bot]

sonic-swss sonic-swss copied to clipboard

[pfc] Introduce pfc config bitmask to track user-configured pfc enabled TCs on physical port

vs tests

test case 1: Clear PFC enable bits on a physical port

test case 2: Separate user and wd PFC enable bits on a physical port

Test fail point 1

Test fail point 2

Test fail point 1

Test fail point 2

sonic-swss
sonic-swss copied to clipboard