spekframework
afterEach - support triggering off of results
Sometimes I need to do something conditionally in after some test on whether it passed (nothing thrown), failed (AssertionError thrown), or erred (Throwable other than AssertionError thrown). I can't find a way, however, to get the status on a test so that I can clean-up appropriately. How can this be accomplished?
I also sometimes need to do this after a group of tests in which case I would need to be able to inspect the statuses for all of the it assertions after a context, on, given, etc.
if (testPassed) sauceREST.jobPassed(jobId) else sauceREST.jobFailed(jobId)
hmmmm, now that I write it out I wonder if an inline function to wrap my assertions would be better. e.g.:
inline fun DescribeBody.sauceIt(description: String, assertions: () -> Unit) {
it(description) {
try {
assertions()
sauceREST.jobFailed(jobId)
} catch (t: Throwable) {
sauceREST.jobFailed(jobId)
throw t
}
}
}
but then I have to write sauceIt instead of it...
IMHO that should be the responsibility of your build process not your tests. This raises several red flags for me:
- What happens when
sauceREST.jobFailed(jobId)fails? It's most definitely not a test failure. - Say I want to run the tests locally, do I need to have access to that service?
Spek uses JUnit, there should be a result/reports file somewhere, depending on the build tool you're using.
With TestNG I would do this with an AfterMethod and/or ITestListener to "extend" the framework. That way I can run my tests independent of a build process. Not all tests/specifications are unit tests and not all tests/specifications are tied to build processes. Even so, I do agree that this shouldn't be part of the test but in order for it not to be I need some kind of hook. I can call JUnit programmatically and then consume the result/reports but then I loose IDE integration. I'd prefer to have some way to integrate with the specifications framework if possible.
And yes, when running Sauce Labs tests you have to have access to its REST API in order to run tests as well as update their statuses.
I guess this falls under extending Spek, which is currently impossible. I suggest using your workaround for now, we will definitely look into this after 1.0.