kotest
Data-driven testing (`withData`) with `BehaviourSpec` doesn't appear to work
Please consider asking your question on our slack channel #kotest or in stack overflow (and tagging with kotest). The github issue tracker is for bugs and feature requests.
I'm trying to combine some data driven tests with other tests within a single Given context under a BehaviourSpec. However, I can't seem to make it work.
The docs say: "Data tests can only be defined at the root or in container scopes. They cannot be defined inside leaf scopes." (https://kotest.io/docs/framework/datatesting/data-driven-testing.html). However, when trying to define a withData block inside a Given block (isn't this a root or container scope?), I can't get it to compile.
class MyTests :
BehaviorSpec({
Given("Some Numbers") {
withData(1, 2, 3) { (a, b, c) ->
{ When("Adding") { Then("Correct!") { a + b + c shouldBe 6 } } }
{ When("Multiplying") { Then("Yes!") { a * b * c shouldBe 6 } } }
}
When("doing something else") { Then("hurray!") { 4 + 5 shouldBe 9 } }
}
}
Gives compile errors of:
Destructuring declaration initializer of type Int must have a 'component1()' function
Destructuring declaration initializer of type Int must have a 'component2()' function
Destructuring declaration initializer of type Int must have a 'component3()' function
Suspension functions can be called only within coroutine body
Obviously, the simple FunSpec example works fine:
class MyTests :
FunSpec({
context("Given some numbers, they're all positive") {
withData(1, 2, 3) { i -> run { i shouldBeGreaterThan 0 } }
}
})
What are the rules on where withData can be used? Is there a way to do data-driven testing with BehaviourSpec?
withData(1,2,3) { ... }
Is going to provide you with 3 invocations, each with a single parameter, like what you did in the fun spec example.
This works:
class MyTests : BehaviorSpec({
Given("Some Numbers") {
withData(1, 2, 3) { a ->
When("Multiplying") { Then("Yes!") { a shouldBe 6 } }
}
When("doing something else") { Then("hurray!") { 4 + 5 shouldBe 9 } }
}
})
I'm still struggling to get multiple tests within the same withData context to execute. If I amend your example to include two When clauses within the withData, it seems to only actually run one of them:
class MyTests :
BehaviorSpec({
Given("Some Numbers") {
withData(1, 2, 3) { a ->
When("Multiplying by 2") { Then("should be a multiple of 2") { (a * 2) % 2 shouldBe 0 } }
When("Checking sign") { Then("Should be positive") { a shouldBeGreaterThan 0 } }
}
}
})
Given: Some Numbers
1
When: Checking sign
Test Then: Should be positive PASSED
2
When: Checking sign
Test Then: Should be positive PASSED
3
When: Checking sign
Test Then: Should be positive PASSED
SUCCESS: Executed 3 tests in 4.2s
This should execute 6 tests, no?
If I switch the assertions to fail, I do get 6 failures:
Given: Some Numbers
1
Test Then: should be a multiple of 2 FAILED
io.kotest.assertions.AssertionFailedError: expected:<1> but was:<0>
When: Checking sign
Test Then: Should be positive FAILED
java.lang.AssertionError: 1 should be > 10
Caused by: java.lang.AssertionError: 1 should be > 10
2
Test Then: should be a multiple of 2 FAILED
io.kotest.assertions.AssertionFailedError: expected:<1> but was:<0>
When: Checking sign
Test Then: Should be positive FAILED
java.lang.AssertionError: 2 should be > 10
Caused by: java.lang.AssertionError: 2 should be > 10
3
Test Then: should be a multiple of 2 FAILED
io.kotest.assertions.AssertionFailedError: expected:<1> but was:<0>
When: Checking sign
Test Then: Should be positive FAILED
java.lang.AssertionError: 3 should be > 10
Caused by: java.lang.AssertionError: 3 should be > 10
FAILURE: Executed 6 tests in 3.7s (6 failed)
6 tests completed, 6 failed
> Task :test FAILED
This could be an issue with gradle's reporting of test counts, but this is what I see:
for
class MyTests : BehaviorSpec() {
override fun concurrency(): Int = 10
init {
Given("Some Numbers") {
withData(1, 2, 3) { a ->
When("Multiplying by 2") {
Then("should be a multiple of 2") { (a * 2) % 2 shouldBe 0 }
}
When("Checking sign") {
Then("Should be positive") { a shouldBeGreaterThan 0 }
}
}
}
}
}
Hmm. I'm seeing the same thing in Intellij (with the kotest plugin):
But if I force a failure on the Multiplying by 2 test, I see the failues:
If I swap the order of the tests, the displayed switches. Seems like it's the last statement in the block that carries the result of the test.