certification-tool icon indicating copy to clipboard operation
certification-tool copied to clipboard

Published 20 hours ago • verified publisher icon project-chip

[Bug] Manual operation in the python testcases need to be implemented in the TH.

Open Saravana-kr22 opened this issue 1 year ago • 4 comments

Describe the bug

The SDK python testcases which has the input() are executing for long time without an end.

For exapmle: while running the TC-RVCRUNM-2.1 , The testcase is neither passed or failed it is in execution for long time.

Expected behavior

The UI prompt needs to be added for these testcases in their respective test-steps to perform the manual operation in the DUT, and then continue the execution.

Saravana-kr22 avatar Jan 30 '24 10:01 Saravana-kr22

HI @cecille, It seems this was related what was discussed in the last meeting

rquidute avatar Jan 30 '24 17:01 rquidute

As per the lastest TH version: Version: 2.10-beta3+spring2024 Sha: 1fd1429 , Below are the list of test cases that requires this implemntation to validate in the TH.

These testcases are execpting to provide input to the script. but TH didn't have any method to provide input to these scripts

  • [ ] TC-OPSTATE-2.1
  • [ ] TC-OPSTATE-2.2
  • [ ] TC-OPSTATE-2.3
  • [ ] TC-OPSTATE-2.4
  • [ ] TC-OPSTATE-2.5
  • [ ] TC-OVENOPSTATE-2.1
  • [ ] TC-OVENOPSTATE-2.2
  • [ ] TC-OVENOPSTATE-2.3
  • [ ] TC-OVENOPSTATE-2.4
  • [ ] TC-OVENOPSTATE-2.5
  • [ ] TC-RVCCLEANM-2.1
  • [ ] TC-RVCCLEANM-2.2
  • [ ] TC-RVCOPSTATE-2.1
  • [ ] TC-RVCOPSTATE-2.3
  • [ ] TC-RVCOPSTATE-2.4
  • [ ] TC-RVCRUNM-2.1
  • [ ] TC-RVCRUNM-2.2

Saravana-kr22 avatar Feb 23 '24 12:02 Saravana-kr22

Hi @Saravana-kr22, @cecille,

We've just opened 2 PRs to handle this issue:

  • TH backend PR: https://github.com/project-chip/certification-tool-backend/pull/73
  • SDK PR: https://github.com/project-chip/connectedhomeip/pull/32299

These PRs add the support for getting the input, but the python tests still need to be updated. The SDK PR updates one of the tests as an example.

ccruzagralopes avatar Feb 23 '24 21:02 ccruzagralopes

@cecille Still this issue is not resolved in TH Version: v2.11-beta2+fall2024 Sha: bbeda31.

For Example: TC-RVCRUNM-2.2 Is still failing. TC-RVCRUNM-2.2.log

Ashwinigrl avatar Aug 06 '24 12:08 Ashwinigrl

Hello @Ashwinigrl ,

Running here the TC-RVCRUNM-2.2 we are able to see the prompt request on UI. Please see the image attached image

Is the error you are seeing related to the prompt display? Or is it an error in the normal test execution?

If it is the prompt display, could you please share both the project config and the pics files you are using so we can replicate it here?

hiltonlima avatar Aug 29 '24 14:08 hiltonlima

@hiltonlima Re verified the Test case in TH Version: v2.11-beta2+fall2024 Sha: bbeda31 I am able to see the prompt request on UI. Hence closing the issue. Thanks!

Ashwinigrl avatar Aug 30 '24 06:08 Ashwinigrl