spock
spock copied to clipboard
spockframework
Could we have multi-dimensional where clauses?
Would it be possible to express multi-dimensional test data by allowing a sequence of two or more where clauses?
package com.example
import spock.lang.Specification
class ComplexProblemTest extends Specification {
@Unroll
def 'it will always exceed 0: #description'(String description, int w, int x, int y, int z) {
expect:
(w + x) * (y + z) > 0
where:
description | w | x | y | z
'w 01 series' | 0 | 1 | -1 | 5
'w 01 series' | 0 | 1 | -7 | 8
'w 01 series' | 0 | 1 | 0 | 1
'w 01 series' | 0 | 1 | 1 | 0
'w 02 series' | 0 | 2 | -1 | 5
'w 02 series' | 0 | 2 | -7 | 8
'w 02 series' | 0 | 2 | 0 | 1
'w 02 series' | 0 | 2 | 1 | 0
'w 13 series' | 1 | 3 | -1 | 5
'w 13 series' | 1 | 3 | -7 | 8
'w 13 series' | 1 | 3 | 0 | 1
'w 13 series' | 1 | 3 | 1 | 0
'w 14 series' | 1 | 4 | -1 | 5
'w 14 series' | 1 | 4 | -7 | 8
'w 14 series' | 1 | 4 | 0 | 1
'w 14 series' | 1 | 4 | 1 | 0
}
}
becomes:
package com.example
import spock.lang.Specification
class ComplexProblemTest extends Specification {
@Unroll
def 'it will always exceed 0: #description'(String description, int w, int x, int y, int z) {
expect:
(w + x) * (y + z) > 0
where:
y | z
-1 | 5
-7 | 8
0 | 1
1 | 0
where:
description | w | x
'w 01 series' | 0 | 1
'w 02 series' | 0 | 2
'w 13 series' | 1 | 3
'w 14 series' | 1 | 4
}
}
Huh? Why? Okay, I have some tests where I have an invariant assertion, like "the computation is greater than 0" or "the call throws a ValidationException", and the test data consists of two or more groups of data that I am manually building out a where table with all the permutations.
Given that I am a totally error-prone human and also so so so so so lazy, it'd be cool if Spock could do the permutations for me. I think it would be possible to do multi-dimensional where and still enforce that the full set of variables always appear in the test method's parameter list. That validation would, I think, help the user grok the proper (limited) utility of multi-dimensional where as part of their ongoing learning curve for Spock.
Finally, I speculate it would be possible to do that interesting array subscript thing you do with the test method name when it doesn't change due to templated strings when you @Unroll over a multi-dimensional where.
So, in the @Unroll'ed names in the multi-dimensional where example above, you could know by the variables used that there's a "partially templated" test method name. And, you could name the test function permutations in the following way to make it easier for the user to pinpoint which combination of input data has a failure condition when they do fail. For example:
it will always exceed 0: w 01 series[0] PASSED
it will always exceed 0: w 01 series[1] PASSED
it will always exceed 0: w 01 series[2] PASSED
it will always exceed 0: w 01 series[3] PASSED
it will always exceed 0: w 02 series[0] PASSED
it will always exceed 0: w 02 series[1] PASSED
it will always exceed 0: w 02 series[2] PASSED
it will always exceed 0: w 02 series[3] PASSED
it will always exceed 0: w 13 series[0] PASSED
it will always exceed 0: w 13 series[1] PASSED
it will always exceed 0: w 13 series[2]: [Condition not satisfied: ...
it will always exceed 0: w 13 series[3] PASSED
it will always exceed 0: w 14 series[0] PASSED
it will always exceed 0: w 14 series[1] PASSED
it will always exceed 0: w 14 series[2] PASSED
it will always exceed 0: w 14 series[3] PASSED
Thanks for Spock! It makes my Outside-In TDD work bearable and helps me put food on the table!
You know that you don't need to use tables? You can also otherwise calculate the data providers. And you can use any Groovy feature and any library and any own code to do so.
And if you unroll a method with some pattern, it is your responsibility to make the names unique. (at least currently)
Look at this which produces exactly the use-cases you wanted and also has distinguishable names, that are not only unique, but also meaningful and not simply counted from 0 to 3:
@Unroll
def 'it will always exceed 0: w #w#x series (y: #y, z: #z)'() {
expect:
(w + x) * (y + z) > 0
where:
[w, x, y, z] <<
[
[
[0, 1],
[0, 2],
[1, 3],
[1, 4]
],
[
[-1, 5],
[-7, 8],
[ 0, 1],
[ 1, 0]
]
]
.combinations()
*.flatten()
}
That is really slick! I'm not yet so savvy with groovy list-fu. It still
would be nifty to have the DSL do more with where.
On Fri, Dec 6, 2019, 7:00 PM Björn Kautler [email protected] wrote:
You know that you don't need to use tables? You can also otherwise calculate the data providers. And you can use any Groovy feature and any library and any own code to do so.
And if you unroll a method with some pattern, it is your responsibility to make the names unique. (at least currently)
Look at this which produces exactly the use-cases you wanted and also has distinguishable names, that are not only unique, but also meaningful and not simply counted from 0 to 3:
@Unrolldef 'it will always exceed 0: w #w#x series (y: #y, z: #z)'() { expect: (w + x) * (y + z) > 0 where: [w, x, y, z] << [ [ [0, 1], [0, 2], [1, 3], [1, 4] ], [ [-1, 5], [-7, 8], [ 0, 1], [ 1, 0] ] ] .combinations() *.flatten()}
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/spockframework/spock/issues/1062?email_source=notifications&email_token=ACOGQYBSVJ3K5LB6MGHSGGLQXMGVPA5CNFSM4JW7OXYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGF4GAY#issuecomment-562807555, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOGQYEJF2YMZVRTSLIYYYDQXMGVPANCNFSM4JW7OXYA .
I think extending the data table syntax would add confusion, there are other requests for just joining multiple tables into one, so that you could split long lines into multiple tables.
Test should be easy to read, and if possible intuitively to understand whats happening. This feature does not satisfy those requirements IMHO.
Using the groovy collection methods, like @Vampire has shown you, is way more powerful and lets you do other things like permutations, filtering and so on.
Just leaving my 2 cents here, but what @woldie proposes here is one of 2 things I wish I had in Spock and I was browsing Spock Issues today specifically looking to see if they were being talked about.
Having a way to cleanly add another dimension to tests would be fairly helpful. Sure you can use combinations() (and I have used it before), but it doesn't feel as clean as having a second table after where that combines the two. Looking through some of my companies' code, I see some examples of where this could be applied. A simplified example here:
@Unroll
def 'an #account_situation account will receive a #code when updating the #start_state state to #end_state'() {
expect:
foo(account_id, start_state, end_state) == code
where:
account_id | account_situation | code | start_state | end_state
admin() | 'admin' | 200 | unread | read
admin() | 'admin' | 200 | unread | hidden
admin() | 'admin' | 200 | read | unread
admin() | 'admin' | 200 | read | hidden
admin() | 'admin' | 200 | hidden | read
admin() | 'admin' | 200 | hidden | unread
random() | 'random' | 403 | unread | read
random() | 'random' | 403 | unread | hidden
random() | 'random' | 403 | read | unread
random() | 'random' | 403 | read | hidden
random() | 'random' | 403 | hidden | read
random() | 'random' | 403 | hidden | unread
account() | 'user' | 404 | unread | read
account() | 'user' | 404 | unread | hidden
account() | 'user' | 404 | read | unread
account() | 'user' | 404 | read | hidden
account() | 'user' | 404 | hidden | read
account() | 'user' | 404 | hidden | unread
}
Could be transformed into:
@Unroll
def 'an #account_situation account will receive a #code when updating the #start_state state to #end_state'() {
expect:
foo(account_id, start_state, end_state) == code
where:
account_id | account_situation | code
admin() | 'admin' | 200
random() | 'random' | 403
account() | 'user' | 404
combination:
start_state | end_state
unread | read
unread | hidden
read | unread
read | hidden
hidden | read
hidden | unread
}
The combination block could just be syntactic sugar to combinations()*.flatten() as suggested above. Using the above 2 tables would be a lot easier to write and edit than dealing with multi-dimentional arrays:
@Unroll
def 'an #account_situation account will receive a #code when updating the #start_state state to #end_state'() {
expect:
foo(account_id, start_state, end_state) == code
where:
[account_id, account_situation, code, start_state, end_state] <<
[
[
[admin(), 'admin', 200],
[random(), 'random', 403],
[account(), 'user', 404]
],
[
[unread, read],
[unread, hidden],
[read, unread],
[read, hidden],
[hidden, read],
[hidden, unread]
]
]
.combinations()
*.flatten()
}
Side note - you could argue this test should be split up into 3 new tests (and I might agree), but in the actual code there's some setup happening and duplicating the tests in that case will look fairly unclean and bloated.
You could put the to-be-duplicated code in a helper method and call that from the three test methods.
While we can ignore certain iterations with @IgnoreIf/@Requires, it might be helpful to have a filter: block that would completely remove any combinations that we don't want.