Let ignoring scenarios using multiple @ignore tags.

[farost]

Right now the library supports only ignoring scenarios using the @ignore tags. However, it is a practice for other BDD libraries to have a configurable list of tags to ignore. For example, Java's Cucumber is fully configurable for each test: tags = "not @ignore and not @ignore-custom and not @ignore-custom-something-else-whatever-you-want"

It is extremely useful for large projects. For example, when we have a number of clients for a single server, and we want to ensure that all our clients work identically. However, we want to temporarily disable some of the scenarios for specific clients, creating @ignore-client1 and @ignore-client2.

As adding such scalable feature requires time, I'd suggest to at least give developers a way to use several different tags.

• Ideally: send a list of tags to be ignored as arguments.
• Faster: ignore all scenarios with tags @ignore*********.

[ttutisani]

My view:

• List of tags to ignore as arguments - yes.
• Ignore all tags based on tag name pattern - no. This will create a loosely-defined, implicit convention, which I always try to avoid.

[ttutisani]

Also, in some simple cases, organizing tests using tags can help. Those can already be passed in as arguments: https://github.com/ttutisani/Xunit.Gherkin.Quick/blob/master/docs/tags.md

[farost]

@ttutisani Maybe just a reversed "--filter" would be enough! Like "all except of ".

[ttutisani]

You made me think 🙂

So, good news, it's already possible because --filter takes an expression as an argument. So, if you want to ignore all tests annotated with @ignore-xxx tags, that means you want to run all tests that have tags that don't contain ignore. You write that like this:

filter Category!~@ignore

The only problem left is that this is not documented in the tags documentation, which is why this issue must stay open until it's documented.

Sources:

• dotnet test command-line reference explains --filter <EXPRESSION> argument.
• <EXPRESSION> part of it is explained on this page: https://learn.microsoft.com/en-us/dotnet/core/testing/selective-unit-tests?pivots=xunit#syntax

[farost]

I've tested the --filter flag. Here are my thoughts:

• It filters out tests from consideration completely, so the filtered tests are not shown in the stats of "Skipped", which is a shame;
• The runner has troubles with ! and Bash. ! is a reserved symbol for Bash, so we need to escape it \!, but then the runner says that something is bad with my pattern: [xUnit.net 00:00:00.6207380] Xunit.Gherkin.Quick.ProjectConsumer: Exception filtering tests: Incorrect format for TestCaseFilter Error: Invalid Condition 'Category!~@ignore'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.;
• It is a dotnet CLI flag, and it's not scalable for other builders (for example, I use Bazel, and its rules either don't use dotnet under the hood (which is a surprise) or don't pass the args to it (which is a double surprise).

So any other option would be still helpful, I guess =)