project-chip
[Bug] Many errors when starting tests
Describe the bug
There is a recurring bug where the TH will just fail to start a test with no other information. Re-running the same test with no changes often results in a pass. This is quite disruptive to the testing process. ATLs seem to be aware of this, and advise never running more than one test in a test run since you're almost guaranteed to have a failure in at least one test. You will then need to go back and re-pick the individual failures.
Steps to reproduce the behavior
Run many tests.
Expected behavior
Tests should not fail to start
Log files
PICS file
No response
Screenshots
No response
Environment
No response
Additional Information
No response
@cecille do you have a high-incidence example to share with us (a project config and test selection that most of the times reproduces the problem)?
no it appears random. But pick any set of tests with > 1 test and you will probably see it.
@antonio-amjr see if you can reproduce this. May be fruitful to try to reproduce before and after the change to fix the long delays...
@antonio-amjr see if you can reproduce this. May be fruitful to try to reproduce before and after the change to fix the long delays...
Ok, will do!
Unfortunately, my team was experiencing some side effects from the solution merged above. For some reason, the problem doesn't affected my environment.
Until a better solution with no side effect is proposed, the issue will be maintained open and the solution was reverted with https://github.com/project-chip/certification-tool-backend/pull/166