cucumber-jvm

cucumber-jvm copied to clipboard

Hi, we're using ConcurrentEventListener to access Step text via TestStepStarted event listener. These steps can later be logged to a file and attached to the failed test in the Allure report. Unfortunately, it does not work as expected.

👓 What did you see?

All steps for given scenario are being logged, even the ones that occur after the scenario failure.

##[SECTION]	 Starting scenario d950c44e-1029-40ae-8896-91c1c72414ec
##[STEP]	 02/29/2024 09:53:11 --- action ---
##[STEP]	 02/29/2024 09:53:11 --- check result ---
##[ERROR]	 02/29/2024 09:53:11 --- Step above is failing! ---
##[STEP]	 02/29/2024 09:53:11 --- rest ---
##[STEP]	 02/29/2024 09:53:11 --- of ---
##[STEP]	 02/29/2024 09:53:11 --- the ---
##[STEP]	 02/29/2024 09:53:11 --- scenario ---
##[SECTION]	 Scenario ended with status: FAILED

java.lang.AssertionError: 
Expecting value to be true but was false

	at org.steps.CommonSteps.checkResult(CommonSteps.java:163)
	at ✽.check result(file:///C:/Projects/ConcurrentEventListenerBug/src/test/java/org/features/ConcurrentEventListener.feature:440)


Failed scenarios:
file:///C:/Projects/ConcurrentEventListenerBug/src/test/java/org/features/ConcurrentEventListener.feature:438 # ConcurrentEventListener bug

✅ What did you expect to see?

I expect, that steps after the failed step are not logged, because as name suggests for "TestStepStarted" event - these steps have not yet started because of the failure.

📦 Which tool/library version are you using?

<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-java</artifactId>
    <version>7.15.0</version>
</dependency>
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-testng</artifactId>
    <version>7.15.0</version>
</dependency>
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-picocontainer</artifactId>
    <version>7.15.0</version>
</dependency>

🔬 How could we reproduce it?

Given scenario:

Scenario: ConcurrentEventListener bug
    When action
    Then check result <---- this step is failing
    Given rest
    When of
    And the
    Then scenario

ConcurrentEventListener implementation:

public class StepLogger implements ConcurrentEventListener {

    @Override
    public void setEventPublisher(EventPublisher publisher) {
        publisher.registerHandlerFor(TestStepStarted.class, this::handleTestStepStarted);
    }

    private void handleTestStepStarted(TestStepStarted testStepStartedEvent) {
        if (testStepStartedEvent.getTestStep() instanceof PickleStepTestStep testStep) {
            Step currentStep = testStep.getStep();
            Log.step(currentStep.getText());
        }
    }
}

cucumber.properties:

cucumber.publish.quiet=true
cucumber.snippet-type=camelcase
cucumber.plugin=summary, html:target/reports/cucumber-pretty.html, json:target/reports/cucumber/cucumber.json, \
  io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm, org.utils.StepLogger