Test Run is shown as 'Finished' just after the first test case result is added to it

[mrogachenko-cg]

Describe the bug Test Run is shown as 'Finished' just after the first test case result is added to it. This seems in contrast with the functionality of adding test results with a key TESTOMATIO_PROCEED=1, based at documentation here: https://docs.testomat.io/reference/reporter/pipes/testomatio.html#do-not-finalize-run

Prerequisites

• Playwright and its browsers are installed
• It is configured to use one worker only

e.g. by set workers: process.env.CI ? 1 : 1, in playwright.config

• It is configured to use one browser only

e.g. by commenting out all browsers except

  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },

• Testomatio reporter is installed npm install @testomatio/reporter --save-dev

To Reproduce Steps to reproduce the behavior:

1. Sync tests located in folder "tests" by TESTOMATIO=<secret> npx check-tests@latest Playwright "tests/**/*{.,_}{test,spec}.ts" --typescript --no-detached --sync
2. Create test run by next command, when <#XXX> is some number (to differentiate runs if you have similar) TESTOMATIO=tstmt_Rr9zAx8e8nLOKkWyzotNVd-aXukK2nOU6g1710403033 TESTOMATIO_TITLE='Test Run #XXX' npx start-test-run --launch
3. Remember ID returned from previous step, to use it in next step
4. Run tests, located in example.spec.ts file by TESTOMATIO=<secret> TESTOMATIO_PROCEED=1 TESTOMATIO_RUN=<id_from_previous_step> npx playwright test example.spec Pay attention - TESTOMATIO_PROCEED=1 is used
5. During test execution - observe how test run state is changed in testomat

Expected behavior

• When test started, test run status changed from 'Scheduled' to 'Running' (this works as expected)
• After first case results is added, test run status should be left as 'Running' (this is NOT working)

Screenshots

• Screenshot of the just created test run (Looks like expected - status 'Scheduled')

• Screenshot, when test execution is started, but results are not yet added (Looks like expected - status 'Running')

• Screenshot, when the first test execution is finished and result is transferred to test run (NOT Expected to see it as 'Finished', it should be 'Running')

Additional context This behavior seems like TESTOMATIO_PROCEED=1 key/value is ignored.

[ghost]

Probably this is an issue in reporter:

[ghost]

Issue is not reproduced anymore, probably after this PR https://github.com/testomatio/reporter/pull/341/files

But, it could be just a side-effect of that code, not a real fix for the issue

[DavertMik]

I think issue fixed by server-side changes, not the mentioned PR but anyway it's nice it is not reproducible anymore

[TetianaKhomenko]

Hi @mrogachenko-cg! Can this ticket be closed?

[ghost]

Hi @mrogachenko-cg! Can this ticket be closed?

Yes, please, close the issue.