cucumber-jvm

cucumber-jvm copied to clipboard

🤔 What's the problem you've observed?

I am making my own version of RerunFormatter. I want it to save only failed scenarios with certain tag.

When trying to copy the class:

• NiceAppendable is not public (understandable, don't have to care myself as I use kotlin and don't have to worry about checked exceptions, replacing with regular PrintWriter works just fine for me )
• UTF8OutputStreamWriter is not public (no big problem, I can just use OutputStreamWriter(out, Charsets.UTF_8) )
• PrettyFormatter.relativize(URI) is not public (I have to duplicate this method in my code, the method doesn't call anything non-public so it's simple)

✨ Do you have a proposal for making it better?

I suggest 2 possibilities:

• Extract filter condition for relevant testcases to be rerun into another protected method + remove final keyword from the class. That way I can simply extend this class and modify the functionality.
• Make PrettyFormatter.relativize public so that I can make my own implementation less duplication.

📚 Any additional context?

Our tests are very unreliable because of the test environment, which is quite non-deterministic at times (videogame). We are for example expecting rock to just fall down and stay on the ground in the hole, but in very rare cases, the rock gets "lucky" and jumps away causing the test to fail. Since the failure is quite rare, we tagged those tests and want to consider them failed only if they fail X times in a row. So first time we run all tests, then only failed tagged tests, etc.

Seems like "filtered rerun" is perfect fit. If you think rerun is not the good way to do this, I welcome suggestions.