agent-python-pytest
agent-python-pytest copied to clipboard
Rerun keeps old test_item in progress
Describe the bug
When rerunning the launch using rerun=True
and rerun_of=<launch ID>
there is an extra test item generated and it is breaking the test count.
When I tried to investigate the issue I found out that start_test_item in rerun moves the old test item to in_progress which is expected from the documentation.
The issue is that the old test_item in_progress is never deleted. That creates two total tests where one is still in progress.
Retries of the test are not available. The logs from both runs are united together instead of two retries.
This issue was previously reported but was abandoned. https://github.com/reportportal/agent-python-pytest/issues/335
Screenshot with the wrong number of tests in the launch:
I use Report Portal on-premise build 5.11.0 version.
Steps to Reproduce Steps to reproduce the behavior:
-
create a launch and report the data inside it
-
Create a new launch that is a rerun of the previous launch where some tests that previously failed will pass.
Launch data using API:
response_1715598791496.json
Expected behavior
The initial launch has a correct sum of passed/failed tests.
A new launch is created marked as a rerun of the previous launch.
Test items with the same name, and set of parameters and under the same path are updated to in_progress.
After each test item is finished the test items in the report portal are moved from in_progress to the correct status.
At the end of the launch, the sum of passed/failed tests is equal to the total tests.
Retries of the test are visible as in the official documentation.
Actual behavior
The initial launch works correctly, but in the rerun start_test_item method creates in_progress items that are not deleted and this increases the number of total tests.
Package versions
Include version info of the following packages:
reportportal-client - 5.4.1
pytest-reportportal - 5.2.2
python - 3.9.18
Report Portal on-premise build 5.11.0 version