🐞: Allure Report Displays Multiple Scenarios for Karate callonce & call Invocation

[umitozdemirf]

What happened?

When using callonce in Karate, the Allure report displays the called scenario as a separate test case. This behavior increases the number of scenarios shown in the report, which can be misleading when trying to analyze test results.

Steps to Reproduce: Write a Karate test that uses callonce to call another feature file. Run the test and generate an Allure report. Observe that the scenario invoked by callonce appears as a separate test case. Expected Behavior: The callonce-invoked scenario should ideally:

Be nested under the main test case, or Not appear as a separate test case in the report. Actual Behavior: The callonce-invoked scenario is listed as a separate test case, which creates confusion about the actual number of executed tests.

Questions: Is this the expected behavior for Karate with Allure integration? Are there recommended configurations or workarounds to handle this more effectively? Contribution: If this is a confirmed issue or an enhancement opportunity, I’d like to contribute to finding and implementing a solution.

Thank you.

What Allure Integration are you using?

allure-karate

What version of Allure Integration you are using?

2.22.0

What version of Allure Report you are using?

2.22.0

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[baev]

@umitozdemirf, could you please provide a repository with a minimal example to reproduce the problem? A public repository on GitHub works the best. Thanks

[umitozdemirf]

Allure Karate Example Repository

You can use this repository to reproduce the issue: allure-karate-example-call.

Steps to Reproduce

1. Clone the repository:

   git clone https://github.com/umitozdemirf/allure-karate-example-call.git
   cd allure-karate-example-call
   

2. Run the tests with the following command:

   mvn test -Dkarate.options="--tags @smoke"
   

3. The Allure report will be generated in the target/allure-results directory. To view the report, use the following command:

   allure serve target/allure-results
   

I’m happy to collaborate and provide a fix or enhancement related to the call behavior in Allure reports.

Looking forward to your feedback! 😊

FYI @baev

[umitozdemirf]

Hey @baev,

I figured out a fix for the issue where call and callonce steps were showing up as separate scenarios in Allure reports. Here's what I did:

Problem

• Steps like call and callonce were being treated as separate test scenarios.
• This was inflating the test count in Allure and making the reports harder to read.

Solution

1. Locate the Issue:

   • Found that the issue was in the beforeStep and afterStep methods in the AllureKarate class.

2. Update beforeStep:

   • Added a check to skip steps that start with call or callonce:

     @Override
     public boolean beforeStep(final Step step, final ScenarioRuntime sr) {
         final String parentUuid = (String) sr.magicVariables.get(ALLURE_UUID);
         if (Objects.isNull(parentUuid)) {
             return true;
         }
     
         // Skip 'call' and 'callonce' steps
         if (step.getText().startsWith("call") || step.getText().startsWith("callonce")) {
             LOGGER.info("Skipping Allure for call/callonce: " + step.getText());
             return true;
         }
     
         final String uuid = parentUuid + "-" + step.getIndex();
         final io.qameta.allure.model.StepResult stepResult = new io.qameta.allure.model.StepResult()
                 .setName(step.getText());
         lifecycle.startStep(parentUuid, uuid, stepResult);
     
         return true;
     }
     

3. Update afterStep:

   • Also skipped these steps in afterStep to avoid processing them as separate tests:

     @Override
     public void afterStep(final StepResult result, final ScenarioRuntime sr) {
         final String parentUuid = (String) sr.magicVariables.get(ALLURE_UUID);
         if (Objects.isNull(parentUuid)) {
             return;
         }
     
         final Step step = result.getStep();
     
         // Skip 'call' and 'callonce' steps
         if (step.getText().startsWith("call") || step.getText().startsWith("callonce")) {
             LOGGER.info("Skipping Allure for call/callonce: " + step.getText());
             return;
         }
     
         final String uuid = parentUuid + "-" + step.getIndex();
         final Result stepResult = result.getResult();
     
         final Status status = !stepResult.isFailed()
                 ? Status.PASSED
                 : Optional.of(stepResult)
                         .map(Result::getError)
                         .flatMap(ResultsUtils::getStatus)
                         .orElse(null);
     
         final StatusDetails statusDetails = Optional.of(stepResult)
                 .map(Result::getError)
                 .flatMap(ResultsUtils::getStatusDetails)
                 .orElse(null);
     
         lifecycle.updateStep(uuid, s -> {
             s.setStatus(status);
             s.setStatusDetails(statusDetails);
         });
         lifecycle.stopStep(uuid);
     }
     

4. Test the Fix:

   • After these changes, I tested the setup, and the call and callonce steps now show up as part of the parent scenario, not as separate tests.

Result

• Test counts in Allure are accurate now.
• Reports look much cleaner and more understandable.

Let me know if you'd like to discuss this further!

Cheers 🍻