[🐛 Bug]: onComplete hooks parameter "results" is no clear

[carri747]

Have you read the Contributing Guidelines on issues?

• [X] I have read the Contributing Guidelines on issues.

WebdriverIO Version

8.22.0

Node.js Version

18.15.0

Mode

Standalone Mode

Which capabilities are you using?

capabilities: [
        {
            browserName: 'chrome'
        }
    ],

What happened?

In the official doc in "OnComplete" hooks mention: "results object containing test results" and it's no correct

If you run the project that git that has 2 workers and 3 testcases the result that it's showing in the console and if you define a console.log(resutls) in this hooks is something like that

Spec Files: 0 passed, 2 failed, 2 total (100% completed) in 00:00:08

This is Worker's status and not the number of testacases executed

What is your expected behavior?

results paramater in hooks return the number of testcases and not the number of the workers

How to reproduce the bug.

link

Relevant log output

[chrome 120.0.6099.71 windows #0-1]     -false
[chrome 120.0.6099.71 windows #0-1]     +true
[chrome 120.0.6099.71 windows #0-1]
[chrome 120.0.6099.71 windows #0-1]     at World.<anonymous> (file:///C:/Dev/git/testAllure/features/step-definitions/steps.ts:5:35)
[chrome 120.0.6099.71 windows #0-1]     at Object.run (C:\Dev\git\testAllure\node_modules\@cucumber\cucumber\src\user_code_runner.ts:37:21)
[chrome 120.0.6099.71 windows #0-1]     at Object.run (C:\Dev\git\testAllure\node_modules\@cucumber\cucumber\src\runtime\step_runner.ts:50:41)
[chrome 120.0.6099.71 windows #0-1]     at processTicksAndRejections (node:internal/process/task_queues:95:5)
[chrome 120.0.6099.71 windows #0-1]     at async TestCaseRunner.invokeStep (C:\Dev\git\testAllure\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:137:12)        
[chrome 120.0.6099.71 windows #0-1]     at async TestCaseRunner.runStep (C:\Dev\git\testAllure\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:334:20)
[chrome 120.0.6099.71 windows #0-1]     at async C:\Dev\git\testAllure\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:240:34
[chrome 120.0.6099.71 windows #0-1]     at async TestCaseRunner.aroundTestStep (C:\Dev\git\testAllure\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:170:28)    
[chrome 120.0.6099.71 windows #0-1]     at async TestCaseRunner.runAttempt (C:\Dev\git\testAllure\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:218:7)
[chrome 120.0.6099.71 windows #0-1]     at async TestCaseRunner.run (C:\Dev\git\testAllure\node_modules\@cucumber\cucumber\src\runtime\test_case_runner.ts:188:29)
[chrome 120.0.6099.71 windows #0-1]     at async Runtime.runTestCase (C:\Dev\git\testAllure\node_modules\@cucumber\cucumber\src\runtime\index.ts:89:20)
[chrome 120.0.6099.71 windows #0-1]     at async Runtime.start (C:\Dev\git\testAllure\node_modules\@cucumber\cucumber\src\runtime\index.ts:116:7)
[chrome 120.0.6099.71 windows #0-1]     at async runCucumber (C:\Dev\git\testAllure\node_modules\@cucumber\cucumber\src\api\run_cucumber.ts:125:19)
[chrome 120.0.6099.71 windows #0-1]     at async CucumberAdapter.run (file:///C:/Dev/git/testAllure/node_modules/@wdio/cucumber-framework/build/index.js:186:33)
[chrome 120.0.6099.71 windows #0-1]     at async Runner.run (file:///C:/Dev/git/testAllure/node_modules/@wdio/runner/build/index.js:148:24)


Spec Files:      0 passed, 2 failed, 2 total (100% completed) in 00:00:08

2023-12-14T08:17:58.433Z INFO @wdio/local-runner: Shutting down spawned worker
2023-12-14T08:17:58.696Z INFO @wdio/local-runner: Waiting for 0 to shut down gracefully
2023-12-14T08:17:58.697Z INFO @wdio/local-runner: shutting down

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• [X] I have searched the existing issues

[erwinheitzman]

It clearly states Spec files, not tests. As mentioned previously, there is a difference between that line in your terminal and the hook and I do not see anything wrong here

[carri747]

Ok, perfect but what about if you print the results parameter into the console. I've the same result and in the doc say that it's the results test and not SPECs

[pinim]

+1..... I have to say that the results parameter isn't clear to me, it will be great if it truly contains all the test run results, so everyone will have the ability to play with the results to achieve every behavior according to every particular use case. @erwinheitzman

[wdio-bot]

Thanks for reporting!

We greatly appreciate any contributions that help resolve the bug. While we understand that active contributors have their own priorities, we kindly request your assistance if you rely on this bug being fixed. We encourage you to take a look at our contribution guidelines or join our friendly Discord development server, where you can ask any questions you may have. Thank you for your support, and cheers!

[erwinheitzman]

When stating that the docs say something different, can you share a link to where you read that?

[erwinheitzman]

+1.....

I have to say that the results parameter isn't clear to me, it will be great if it truly contains all the test run results, so everyone will have the ability to play with the results to achieve every behavior according to every particular use case.

@erwinheitzman

What parameter are you talking about? Please be specific

EDIT: I assume you mean the parameter in the hook. What is your suggestion, updating the docs?

[pinim]

+1..... I have to say that the results parameter isn't clear to me, it will be great if it truly contains all the test run results, so everyone will have the ability to play with the results to achieve every behavior according to every particular use case. @erwinheitzman

What parameter are you talking about? Please be specific

EDIT: I assume you mean the parameter in the hook. What is your suggestion, updating the docs?

I meant the parameter described in the title here, the 'results' parameter, in the documentation there is no information about what this parameter object contains in the onComplete hook, So first of all it would be great to see that the documentation describes exactly what this parameter contains at the end of all the tests, Secondly, which is a topic for a separate discussion, is what I proposed, that this object will contain as much detailed information as possible (spec & test cases, retries, failures) so that anyone can extract from it all the information that can be relevant for each use case.

[erwinheitzman]

@pinim thank you for the quick reply, we can certainly update the docs with that information but I don't understand the last part as the results parameter already contains all of that information (passed, finished, failed and retried specs)

If it is just about adding the tests, this is part of the spec (aka afterSuite hook)