Allure Reqnroll: Support for `@ignore` tag

[mrairdon-midmark]

Steps to reproduce

Create a gherkin file with the @ignore tag on it

Expected result

These should show up in the test results as "Skipped"

Actual result

These tests do not show up in the results at all

Additional reference

Would like to see these tests show up as "Skipped" in the report:

https://docs.reqnroll.net/latest/gherkin/gherkin-reference.html#:~:text=Reqnroll%20treats%20the%20%40ignore%20tag%20as%20a%20special%20tag.%20Reqnroll%20generates%20an%20ignored%20test%20method%20from%20scenarios%20with%20this%20tag.

[mrairdon-midmark]

@epszaw open to accepting a PR for this?

[Tiberriver256]

Had Claude 4 dig into this a bit. Felt it was worth sharing:

Investigation Summary

Key Findings

1. How Reqnroll Handles @ignore Tag:

   • Reqnroll generates test code with [Xunit.SkippableFactAttribute(Skip="Ignored")]
   • This causes XUnit to skip the entire test method at the framework level
   • The generated code includes testRunner.SkipScenario() call, but it's never reached because XUnit skips the method entirely

2. Current Allure Integration:

   • Only handles ScenarioFinishedEvent for completed scenarios
   • Missing ScenarioSkippedEvent and ScenarioStartedEvent handlers
   • For @ignore tagged scenarios, no Allure events are fired because the test method never executes

3. Difference Between Ignore Types:

   • @ignore tag: XUnit skips entire test method → No Reqnroll events fired → No Allure result
   • @runtimeignore tag: Test method runs, calls TestIgnore() during execution → Allure result generated

Root Cause Analysis

The fundamental issue is that XUnit's Skip attribute prevents the test method from executing entirely, which means:

• ScenarioInitialize() is never called
• testRunner.SkipScenario() is never called
• No Reqnroll events (including ScenarioSkippedEvent) are fired
• Our ScenarioSkippedEventHandler never gets invoked

Generated Code Analysis

Looking at IgnoreTag.feature.cs, the generated code shows:

[Xunit.SkippableFactAttribute(DisplayName="Should be skipped due to ignore tag", Skip="Ignored")]
public async System.Threading.Tasks.Task ShouldBeSkippedDueToIgnoreTag()
{
    // This entire method body never executes due to XUnit Skip attribute
    string[] tagsOfScenario = new string[] { "ignore" };
    this.ScenarioInitialize(scenarioInfo);
    if (ContainsIgnoreTag(...))
    {
        testRunner.SkipScenario(); // ← Never reached!
    }
    // ...
}

Potential Solutions

Option 1: Hook into XUnit Skip Mechanism

• Intercept XUnit's test discovery/execution pipeline
• Create Allure results for tests marked with Skip attribute
• Pros: Works with existing Reqnroll generation
• Cons: Complex XUnit integration, framework-specific

Option 2: Custom Test Discovery

• Implement custom test discovery that identifies @ignore scenarios
• Generate Allure results during discovery phase
• Pros: Framework agnostic approach
• Cons: Complex implementation, may interfere with test runner

Option 3: Modify Reqnroll Generation (if possible)

• Change how Reqnroll generates code for @ignore scenarios
• Ensure ScenarioInitialize() and SkipScenario() are called even for skipped tests
• Pros: Clean solution, works with existing event system
• Cons: Requires Reqnroll framework changes

Option 4: Post-Test Analysis

• Analyze generated test assembly for skipped tests
• Generate Allure results based on attributes and test metadata
• Pros: Works with existing framework
• Cons: Requires separate analysis step

Key Reference Links

• Reqnroll @ignore documentation
• Reqnroll TagHelper.ContainsIgnoreTag source
• Reqnroll OnScenarioSkippedAsync source