[CERT-TEST-FAILURE] -TC-CNET- 4.9 - Test failed

[KishokG]

Feature Area

Other

Test Case

TC-CNET-4.9

Reproduction steps

1. Built all-clusters-app and chiptool in two seperate RPI
2. Brought up DUT using commnad : ./chip-all-clusters-app --wifi
3. Commission the Controller using ./chip-tool pairing ble-wifi 1 SSID SSID-PWD 20202021 3840
4. Armed the fail-safe : ./chip-tool generalcommissioning arm-fail-safe 900 1 1 0
5. Checked the 1st connected ACCESSPOINT : ./chip-tool networkcommissioning read networks 1 0

[1657733068.835240][7475:7480] CHIP:TOO: Networks: 1 entries [1657733068.835354][7475:7480] CHIP:TOO: [1]: { [1657733068.835434][7475:7480] CHIP:TOO: NetworkID: 47524C50726976617465 [1657733068.835487][7475:7480] CHIP:TOO: Connected: TRUE [1657733068.835535][7475:7480] CHIP:TOO: }

6. Removed the network : ./chip-tool networkcommissioning remove-network hex:47524C50726976617465 1 0 --Breadcrumb 1
7. Now Disarmed the fail-safe : ./chip-tool generalcommissioning arm-fail-safe 0 0 1 0
8. Checked whether the network is reverted to 1st ACCESS POINT : ./chip-tool networkcommissioning read networks 1 0

Expected Outcome:

[1684497703.596592][3941:3943] CHIP:TOO: Networks: 1 entries [1684497703.596640][3941:3943] CHIP:TOO: [1]: { [1684497703.596668][3941:3943] CHIP:TOO: NetworkID: 47524C50726976617465 [1684497703.596697][3941:3943] CHIP:TOO: Connected: TRUE [1684497703.596721][3941:3943] CHIP:TOO: }

Actual Outcome:

[1711019121.312656][7411:7413] CHIP:TOO: Endpoint: 0 Cluster: 0x0000_0031 Attribute 0x0000_0001 DataVersion: 3881988887 [1711019121.312745][7411:7413] CHIP:TOO: Networks: 0 entries [1711019121.313000][7411:7413] CHIP:EM: <<< [E:54187i S:41064 M:121351343 (Ack:118096550)] (S) Msg TX to 1:0000000000000001 [D8F4] [UDP:[fe80::e65f:1ff:fe0f:2878%eth0]:5540] --- Type 0000:10 (SecureChannel:StandaloneAck)

Bug prevalence

Whenever I do this

GitHub hash of the SDK that was being used

d2eab7f07ae9289fe14aa7e13dba7e64904ef80e

Platform

raspi

Anything else?

• TH should be able to connect DUT using 1st ACCESSPOINT with Network Id and connected status as TRUE when the fail-safe is disarmed.

• It appears that an issue arose between the SVE commit 9f6d627e0262e1d023986291948bb4e845be803e and Feb 21st commit 0c3f012c71cf3466ad5fe6d236b47e2bce914c7b. This commit disables WIFI PDC support, which might have caused the test failure.

Failing Test step: 12

Note: No issues were encountered with the SVE commit.

DUT Logs: TC-CNET-4.9(DUT).txt Chip-tool Logs: TC-CNET-4.9.txt

Test plan reference: https://github.com/CHIP-Specifications/chip-test-plans/blob/master/src/cluster/network_commissioning.adoc#tc-cnet-4-9-wi-fi-verification-for-removenetwork-command-dut-server

[cecille]

Does linux have a hook up to swap the wifi network from within the example app?

ie, is this a NEW failure? Has this ever worked on the pi?

[kagami555]

I would like to know if the problem has been solved? The same problem also occurred with the chip-lighting-app I compiled on the linux platform.

[KishokG]

Does linux have a hook up to swap the wifi network from within the example app?

ie, is this a NEW failure? Has this ever worked on the pi?

@cecille It's a new failure. It was worked fine in RPI platform

[kagami555]

Hello, Kishok G do you want to ask, the steps of connecting to wifi in the matter program on Linux are done by other processes. In fact, this is indeed the case in my project. First of all, due to business needs, we need wifi. The management is placed in a process. Secondly, when I use this commit to compile, the action of connecting to wifi will fail. I think that the matter process will not actually be allowed to operate wifi, so I will skip it. Does this have any impact?

@.***

From: Kishok G Date: 2024-04-29 14:01 To: project-chip/connectedhomeip CC: kagami555; Comment Subject: Re: [project-chip/connectedhomeip] [CERT-TEST-FAILURE] -TC-CNET- 4.9 - Test failed (Issue #32663) Does linux have a hook up to swap the wifi network from within the example app? ie, is this a NEW failure? Has this ever worked on the pi? @cecille It's a new failure. It was worked fine in RPI platform — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[kagami555]

This has always been a problem, I never got it working correctly on my pi because of the commit I always used (437c5743ef4eac87ec5f80114d295f71e8a4f0e0)

@.***

From: Kishok G Date: 2024-04-29 14:01 To: project-chip/connectedhomeip CC: kagami555; Comment Subject: Re: [project-chip/connectedhomeip] [CERT-TEST-FAILURE] -TC-CNET- 4.9 - Test failed (Issue #32663) Does linux have a hook up to swap the wifi network from within the example app? ie, is this a NEW failure? Has this ever worked on the pi? @cecille It's a new failure. It was worked fine in RPI platform — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[cecille]

If your device is connected to the operational network BEFORE matter commissioning starts (ie, if the user has a way to set the network credentials outside of matter and this is required for proper functioning of the device) then you should not be including a network commissioning cluster and you should not be using BLE. See spec section 4.3.1 - Commissionable Node Discovery "A commissionable Node that is already connected to an IP-bearing network SHALL only make itself discoverable on the IP network and SHALL use the relevant DNS-SD service (_matterc._udp) described below."

However, I'm not certain how you'd accomplish this in a reasonable way on a production device unless you had a custom commissioning step. ie, fresh out of the box, how would a user give your device wifi credentials? If it's through the matter commissioning process then you need BLE and network commissioning and the device cannot be connected to a wifi network before commissioning starts. If the device needs wifi credentials before matter commissioning, then it's up to whatever mechanism your product designed to get those allocated, and the discussion of the network commissioning cluster and its associated tests is moot.

Note also that 1) the PI os is not vanilla linux and the linux platform may not work for managing the network stack there and 2) I'm not sure we HAVE any production devices running on a pi base right now so you could be breaking new ground here.

[cecille]

@KishokG - neither of the supplied logs show the final arm failsafe command or the network read. Can you please provide full logs.

[andy31415]

I would like to know if the problem has been solved? The same problem also occurred with the chip-lighting-app I compiled on the linux platform.

@kagami555 the examples included with the matter SDK are included as a starting point and built up sufficiently to help us CI and validation, however this is where typical maintenance stops.

Building an actual product goes beyond that. If you need specific fixes, the fastest path is likely for you to investigate the root cause and propose a fix PR.

[KishokG]

@KishokG - neither of the supplied logs show the final arm failsafe command or the network read. Can you please provide full logs.

@cecille Here is the full log of TC-CNET-4.9.

DUT Logs: TC-CNET-4.9(DUT) log.txt Chip-tool Logs: TC-CNET-4.9(Chip-tool) log.txt

[kagami555]

This is the completed log of my recent test, thank you very much

发件人：Kishok G @.> 发送时间：2024年5月10日(星期五) 13:32 @.> @.>; @.> 主　题：Re: [project-chip/connectedhomeip] [CERT-TEST-FAILURE] -TC-CNET- 4.9 - Test failed (Issue #32663) @KishokG <https://github.com/KishokG > - neither of the supplied logs show the final arm failsafe command or the network read. Can you please provide full logs. @cecille <https://github.com/cecille > Here is the full log of TC-CNET-4.9. DUT Logs: TC-CNET-4.9(DUT) log.txt <https://github.com/project-chip/connectedhomeip/files/15270673/TC-CNET-4.9.DUT.log.txt > Chip-tool Logs: TC-CNET-4.9(Chip-tool) log.txt <https://github.com/project-chip/connectedhomeip/files/15270675/TC-CNET-4.9.Chip-tool.log.txt > — Reply to this email directly, view it on GitHub <https://github.com/project-chip/connectedhomeip/issues/32663#issuecomment-2103899713 >, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A46MA2TAG3Z3SGADVPD6XKDZBRLVTAVCNFSM6AAAAABFBIINVCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBTHA4TSNZRGM >. You are receiving this because you were mentioned.Message ID: @.***>

[cecille]

@KishokG - can you also confirm whether this test works as expected for you on a normal wifi device? ex. esp32?

[cecille]

Can you also confirm whether the PI is joined to the wifi network before you issue the pairing command?

[KishokG]

Can you also confirm whether the PI is joined to the wifi network before you issue the pairing command?

@cecille Yes. Both(Controller and DUT) RPI's are connected to the wifi network before commissioning.

[KishokG]

@KishokG - can you also confirm whether this test works as expected for you on a normal wifi device? ex. esp32?

I ran TC-CNET-4.10 tc in Thread platform(NRF52840-DK) and the test is passing without any issues.

Note: TC-CNET-4.9 and 4.10 both test cases has same functionality.

Here is the TC-CNET-4.10 Log: TC-CNET-4.10(Chip-tool) log.txt

[cecille]

4.10 is different here because you won't have the problem of having a pre-connected PAN before commissioning.

[KishokG]

@cecille

@KishokG - can you also confirm whether this test works as expected for you on a normal wifi device? ex. esp32?

• Tested TC-CNET-4.9 in ESP32 platform and it's PASSING.

Here is the logs:

• TC-CNET-4-9-DUT.log
• TC-CNET-4-9-chip-tool.log

[KishokG]

Now the test is PASSING in this commit: 51e4846460a6760800fc53206dadda26a255f932 (Tested in RPI platform).

Here is the Logs:

• TC-CNET-4.9(Chip-tool) log_19_06_2024.txt
• TC-CNET-4.9(DUT) log_19_06_2024.txt

[KishokG]

Tested in RPI platform and Again the test is FAILING in this commit: d0d91272068f267cf880f9d56787ca28da885673

Here is the log: TC-CNET_4.9_NonTE2.txt