[1.1] Moving pairing / commissioning outside of test suite run

[raju-apple]

From chip-certification-tool created by raju-apple: CHIP-Specifications/chip-certification-tool#584

Will also unblock the following 2 issues :

• [ ] Sorting TCIDs
• [ ] Removing fabric after each test run

[raju-apple]

Proposed solution

Since Test Harness has all the needed information in the project's configuration, we can ask the API caller to provide us with the project ID and then we get access to all the information we need for pairing.

• Create the API /api/v1/matter/commissioning/commission/{project_id} to pair with the device using the project's configuration information (pairing mode, setup code, discriminator, ...).
• Create the API /api/v1/matter/commissioning/decommission/ for decommissioning.
• At the start of a test suite run, TH will verify if the device is already commissioned. If the device is not commissioned yet, TH will do it automatically. (For this to be possible, we need a way of knowing if the device is paired or not. We don't know how to do it yet).
• Add a field to the project's configuration to indicate whether the test suite should decommission the device at the end of the run or not (default value is true).
• At the end of a test suite run, TH will check for the new field in the project's configuration to decommission the device or not.
• Since TH only supports one pairing at a time, if the commisioning API is called for a device and then called for another one later, TH will do the following:
  • Check if the commissioning parameters are the same as the ones used in the last commissioning.
    • If they are the same:
      • Verify if the connection still works. If it doesn't, send pairing command to chip-tool.
    • If they are different
      • Decommission previous device and commission the new one.
• Create class to handle commissioning (Commissioner). Both the new APIs and the test suite will use this new class.

Steps

The development will be done in separate PRs:

1. Create Commissioner class and refactor test suite to use it. (ETA: open PR until 01/13)
2. Create the API endpoints and add new field in the project's configuration. (ETA: TBD)
3. Add check to verify if the device is already commissioned or not. (ETA: TBD)
4. Store commissioning credentials and verify it when commissioning. (ETA: TBD)
5. Sync with UI team.

[raju-apple]

Updating ETAs:

1. Create Commissioner class and refactor test suite to use it. (ETA: open PR until 01/13)
2. Create the API endpoints (ETA: open PR until 01/17)
3. Add check to verify if the device is already commissioned or not and add new field in the project's configuration.. (ETA: open PR until 01/24)
4. Store commissioning credentials and verify it when commissioning. (ETA: open PR until 01/27)
5. Parse log to return correct HTTP error code for commissioning. (ETA: TBD)
6. Sync with UI team.

[raju-apple]

The first try to solve this issue has caused problems for simulated tests, so it has been reverted and we took a step back to re-think the solution

[raju-apple]

The new solution consists of changing the way we handle the chip tool container life cycle. Instead of creating it and destroying it at every test suite setup and cleanup, we'll handle that just like the other containers (frontend, backend, db, proxy): using the start.sh and start-dev.sh scripts. There are draft PRs for the first part of this solution, but they are on hold for now because we're waiting on the integration of the python runner in TH, which will change the way we interact with chip tool.

[fabiowmm]

On further analysis we dropped this feature.