SpecFlow icon indicating copy to clipboard operation
SpecFlow copied to clipboard

Provide reason with @ignore tag

Open carlin-q-scott opened this issue 8 years ago • 14 comments

It would be great if we could provide a reason with the ignore tag that would show up in the test report. I seem to recall that some tags allow text already so this shouldn't be too hard to implement.

This would help my team which is practicing Trunk Based Development; It's important for us because we only have one set of system tests for a variety of services that can be deployed independently at any time.

carlin-q-scott avatar Feb 22 '16 17:02 carlin-q-scott

I seem to recall that some tags allow text already so this shouldn't be too hard to implement.

:heart_eyes: You thinkin' what I'm thinkin'? :heart_eyes:

darrencauthon avatar Feb 22 '16 17:02 darrencauthon

That I should implement this is put in a pull request?

carlin-q-scott avatar Mar 04 '16 04:03 carlin-q-scott

that seems like it could be a good idea...

On Fri, Mar 4, 2016 at 4:09 AM, Carlin Scott [email protected] wrote:

That I should implement this is put in a pull request?

— Reply to this email directly or view it on GitHub https://github.com/techtalk/SpecFlow/issues/555#issuecomment-192090408.

samholder avatar Mar 04 '16 09:03 samholder

which flavour of test generation do you use, as the support for this is probably dependent of how the testing framework implements its Ignore attribute

samholder avatar May 13 '16 08:05 samholder

I'm using SpecRun so I would most likely need to throw a NotImplementedException(reasonString). At least I think that's the exception that is thrown when a test isn't done.

carlin-q-scott avatar May 14 '16 15:05 carlin-q-scott

I'm confused - what does a NotImplementedException have to do with an ignored scenario? As I understand it, a NotImplementedException is something you throw when you haven't yet implemented a particular method. @carlin-q-scott Could you please clarify what you mean?

dirkrombauts avatar May 14 '16 16:05 dirkrombauts

If you are using the SpecFlow+Runner (SpecRun) the test is never been executed if it has an ignore tag. It reports back, that this test is skipped. That happens in VS via an API call to the test explorer.

As it concerns SpecFlow+Runner i will put this on our list of feature requests.

SabotageAndi avatar May 14 '16 16:05 SabotageAndi

@dirkrombauts I was figuring that throwing that exception was the best way to bubble up a reason for the skipped test.

carlin-q-scott avatar May 17 '16 16:05 carlin-q-scott

@carlin-q-scott I think I can see where you are coming from ... but what you describe is a run-time behaviour, and the ignore tag is converted into an Ignore attribute while you type the feature file. And as Andi said, a scenario with an Ignore attribute isn't run at all.

So anyway, Andi has taken it onboard with the SpecFlow+ feature requests.

dirkrombauts avatar May 18 '16 07:05 dirkrombauts

Hi We are using Specflow with MsTest, We need to ignore scenario with custom reason like bellow

@ignore"Test_purpose" Scenario: Open Edit From MainMenu Given the application is default state When I open the program editor page Then the program editor page should be shown in feature file

The designer file created like bellow

[Microsoft.VisualStudio.TestTools.UnitTesting.TestMethodAttribute()] [Microsoft.VisualStudio.TestTools.UnitTesting.DescriptionAttribute("Open Edit From MainMenu")] [Microsoft.VisualStudio.TestTools.UnitTesting.TestPropertyAttribute("FeatureTitle", "SmokeTests")] [Microsoft.VisualStudio.TestTools.UnitTesting.IgnoreAttribute()] [Microsoft.VisualStudio.TestTools.UnitTesting.TestCategoryAttribute("Test_purpose")] public virtual void OpenEditFromMainMenu() { TechTalk.SpecFlow.ScenarioInfo scenarioInfo = new TechTalk.SpecFlow.ScenarioInfo("Open Edit From MainMenu", new string[] { "ignore", "Test_purpose"}); #line 10 this.ScenarioSetup(scenarioInfo); #line 11 testRunner.Given("the application is default state", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Given "); #line 12 testRunner.When("I open the program editor page", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "When "); #line 13 testRunner.Then("the program editor page should be shown", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Then "); #line hidden this.ScenarioCleanup(); }

here is there any possibility to pass the ignore message like bellow in the designer file [Microsoft.VisualStudio.TestTools.UnitTesting.IgnoreAttribute("Test_purpose")]

if not, please provide more details about ignoring the scenario or feature with custom message please provide any one of the framework like NUnit, MsTest and XUnit will support ignore with custom reason, Any special tools we have to install to enable this feature

SubbuPagadala avatar May 30 '18 09:05 SubbuPagadala

Hi guys, I would love to see this working with NUnit too. The @Ignore attribute with a reason gets allways interpreted as a categoryattribute. If you add spaces to it, it gets even worse: A sepearation into seperate category attributes.

StarWars999123 avatar Aug 03 '18 08:08 StarWars999123

@SabotageAndi Any update on this?

godrose avatar Apr 10 '20 12:04 godrose

Nope. The big problem here is, that Spaces aren't allowed in Gherkins for tags. So I had not yet an idea of how to add the reason to a tag.

SabotageAndi avatar Apr 10 '20 13:04 SabotageAndi

@SabotageAndi Use some convention - underscore or @space or something similar. As this is optional it might be better adding this option and replacing the special symbol in the generated code-behind file.

godrose avatar Apr 11 '20 09:04 godrose