reporter API: spec started event

[segrey]

Feature request for IntelliJ integration. Now reporter is notified when a spec execution is finished. It'd be better for user to be see what tests are executed at the moment. It'd be nice to have onSpecStarted(browser, spec) in BaseReporter.

[segrey]

Another benefit is that it would be possible to attach captured messages sent to console.log to correct tree test nodes. Now messages sent to dump or console.log are attached to a random test tree node.

[vojtajina]

:+1:

[dignifiedquire]

@segrey is this still needed and would you be willing to work on this as I probably won't have?

[segrey]

Yes, it's still needed. I'll try to find time to work on this.

[AbdealiLoKo]

I would like this too - would a PR be welcome ? I see this is tagged as help wanted but it's quite old ...

[devoto13]

The PR is still welcome.

Note however that it may be a bit more complex than it looks from the description as besides adding to the BaseReporter it should be added to the communication protocol and to the test framework plugins (karma-jasmine, karma-mocha, etc) to be useful. Consequently, it will either be an ecosystem-wide breaking change requiring major bumps of several plugins or we should come up with an approach how to make it backward compatible when reporter, but not framework supports it or the other way around.

I would suggest starting with the proposal/design/proof of concept, getting a green light on that, and then implementing the PR.

[AbdealiLoKo]

Understood @devoto13 - Let me see how far I can go. I do have some time limitations too - but I'm hopeful I have seen a lot of inconveniences that this will resolve, and a lot of developers getting confused about error logs coming in before the testsuite getting reported etc.

I can definitely check karma-jasmine, but I have never used karma-mocha before. But I generally feel like backward compatibility would be required.

I will spend some time on looking through the code and understanding the current websocket protocol and come up with some ideas on implementation. Would you have any documentation I could read regarding the internals ? Or just going through the code is best

[devoto13]

No, I'm afraid, we don't have much documentation on the internals... This high-level overview is probably the best we've got.

Just to give you some hints:

• We've got this violation of the grep test, so often onSpecComplete is a handler for the emit('spec_complete') event. So keep that in mind when looking for usages.
• You can find events sent by browser by looking for socket.emit in the client code.
• These events are then received by the server code.
• Which in turn produces other events which are received by the reporters.

So you would need to add a method to the client code to send new spec started event, receive it on the server side and pass it to the reporter. Then framework plugins should start using this method to send the event and reporter plugins can make use of it. That's where I'm not sure how we can land it safely... Because somebody can have a reporter expecting this event, but framework which does not send it.

Looking at the karma-jasmine code, I've noticed that they seem to use info event ot pass custom payloads, but they don't seem to be used. Not sure what is going on there.