[💡 Feature]: Listen for `envelope.attachment` in the Cucumber messages event listener for reporters to use

[hammzj]

Is your feature request related to a problem?

In Cucumber v7, a new formatter named Cucumber Messages has been added to replace the old JSON reporter. This addition allows for better streaming of data to reduce memory overhead, but also unifies the approach to attaching metadata and other attachments to Cucumber scenarios.

Necessarily, it replaces the Cucumber JSON formatter currently present within WebDriverIO and is the preferred solution for Cucumber formatters going forward. As such, it would be an excellent addition to WebDriverIO.

As there is already a listener available, we should extend it to listen to the attachment message and utilize it. This would be incredibly beneficial for reporters.

Describe the solution you'd like.

In the @wdio/cucumber-framework cucumberEventListener.ts, there is a listener for the envelope and many of its different messages.

Specifically, there is one object, envelope.attachment, which allows attachments to be added to a step or hook, and look like this:

envelope.attachment = {
  "body": "{\"key\": \"value\"}",
  "contentEncoding": "IDENTITY",
  "mediaType": "application/json",
  "testCaseStartedId": "329",
  "testStepId": "323"
}

When the listener sees attachment, it should add that attachment to the testCaseStepId or the specific envelope.scenario.steps instance. Because this could become a heavy process, it would be nice to either enable this with a new cucumberOpts.allowAttachments option, or determine a way to stream the data asyncronously.

Additionally, allow creation of a cucumber-messages.ndjson from the messages post-feature is completed. All it needs to do is ensure that it can create a JSON to an output directory from the intercepted Cucumber messages.

Describe alternatives you've considered.

I've tried the Cucumber JSON reporter but it has been misbehaving by incorrectly printing Before and After hooks, as well as any steps beginning with *, and not capturing attachments as expected.

Additional context

Right now, envelope.attachment is being treated as an unknown envelope. I have debugged it to see that it goes to the else statement debug log.

Here are all of the JSON messages as defined by Cucumber.

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[christian-bromann]

Thanks for reporting! The Cucumber team has been doing some excellent work recently and in the past and we should make sure to stay up to date with their framework.

Any contributions that improves our Cucumber compatibility are highly appreciated. Please take a look into our contribution guidelines and let us know if you have any questions. Cheers!

[hammzj]

So I've determined that without capturing ITestCaseAttemptData in Cucumber-JS, we don't have a formal way of exposing the attachments out to the hooks and such. It seems like relying on reporters to handle messages and attachments is the best way for now. Plus, emitting these after each event might not be necessary either, but I'm not too familiar with event emitters.

This leads me to think that there should be some consideration that Cucumber's World object is going to be separated from WebDriverIO's cucumber framework, and we won't be able to access things inside of that World without it being configured in this framework.

I don't think there's anything else I can check here. Please let me know if I should just close it out.

[tamil777selvan]

Hi @hammzj, WebdriverIO has now implemented the latest version of Cucumber, which allows the generation of cucumber message / json reports and even addition of attachments to specific steps.

Could you please try it out and let us know if it resolves your use case?

[hammzj]

It's been two years since I reported it so I literally cannot remember what I was doing with this lol. I went with a different reporter for the suite.

If anyone would need to test or add a unit test, it would be around using this.attach within a Gherkin step, as that will add an attachment to the Cucumber World object.

[tamil777selvan]

I've given it a try, and it appears to be working as expected.

[christian-bromann]

Thanks for confirming @tamil777selvan , feel free to close the issue.

[tamil777selvan]

I'm closing this issue as it has been resolved in the latest version. Feel free to open a new issue if you come across any problems related to this.