Double execution with xUnit using Test Explorer

[mocsharp]

SpecFlow Version

3.9.52

Which test runner are you using?

xUnit

Test Runner Version Number

3.9.52

.NET Implementation

.NET 5.0

Project Format of the SpecFlow project

Classic project format using <PackageReference> tags

.feature.cs files are generated using

SpecFlow.Tools.MsBuild.Generation NuGet package

Test Execution Method

Visual Studio Test Explorer

SpecFlow Section in app.config or content of specflow.json

None

Issue Description

When the test project is executed using Test Explorer, each and every test is executed twice.

Immediately after Test Explorer shows complete execution, in the background, the test is executed a 2nd time.

Steps to Reproduce

Using the VS template & the calculate example, update GivenTheFirstNumberIs(...) with the following code

            using var stream = File.CreateText($"C:\\some-directory\\test-{DateTime.Now.Ticks}.txt");
            stream.WriteLine(DateTime.Now.ToString());
            stream.Close();

And remove all the throw new PendingStepException lines. Click Run all test in view in Test Explorer

Expected Behavior:

• one single file is created

Actual Behavior:

• two files are created (2nd file is created within 3-5 seconds after 1st)

Link to Repro Project

No response

[mocsharp]

Update: If there are multiple test cases defined and you selected to run just one test case from the Test Explorer. Upon completing the test case, all of them are executed in the background.

[SabotageAndi]

Please try to create a project where we can reproduce this issue. There has to be something in the project what causing this. I am 99% sure the double execution of a test isn't SpecFlow fault.

[mocsharp]

@SabotageAndi I actually created a new project from scratch using the VS extension template.

[SabotageAndi]

in which VS version?

[mocsharp]

2022 Version 17.1.0

[SabotageAndi]

I followed your step but I couldn't reproduce this. Please put somewhere a project where you can reproduce this issue.

[mocsharp]

The sample project can be found on https://github.com/mocsharp/specflow2560

I have updated Steps to reproduce section to use Test Explorer to run the test.

[SabotageAndi]

@mocsharp I still can't reproduce it. I am using the latest version of VS 2022. Just to check, do you have live unit testing enabled?

[mocsharp]

No, I don't. I'm using the latest version of VS Community which doesn't have the live unit test feature.

Microsoft Visual Studio Community 2022 (64-bit) - Current
Version 17.1.0

[HairyPorker]

I experienced exact same behavior. Might worth mentioning that running test using cli dotnet test run once as expected (only call once).

Additionally, I added Unit Test, project and it have the same issue; I use different solution without Specflow project also resulting the same. I think this is an issue from the Test Explorer runner itself ; not specific to Specflow