spock icon indicating copy to clipboard operation
spock copied to clipboard

Limit data value name length in unrolled test

Open T45K opened this issue 2 years ago • 3 comments

Background

We're using Spock for testing of our product. The other day, we found some tests using large size images in where clause were stacked and didn't finish at all. I investigated the root cause, and it was the following reason.

  1. Spock shows data name and value in test case name when we use where clause.
  2. Even if we use objects which are represented by too long string in where clause, Spock tries to render the name.
  3. Then, stacking will occur because showing too long string takes much time.

Proposal

Limiting data value name length to avoid unintentional stacking.

Sample project

I prepared https://github.com/T45K/spock-stack-sample You can reproduce my result by cloning it and importing into IDE (I'm using IntelliJ IDEA).

On master branch, original Spock is used, and you will find AppTest.stack because test target will be shown in test name will take much time.

On solution branch, I set Spock built in this PR. You will find the test is much faster.

Others

I used 1024 as max length. This is just my preference (it is easy to understand because 2 ^ 10). If you have any recommendation, I'll follow.

Thank you.

T45K avatar Jun 24 '23 13:06 T45K

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (e08fe5c) 78.18% compared to head (ba306d6) 78.18%.

Additional details and impacted files
@@            Coverage Diff            @@
##             master    #1702   +/-   ##
=========================================
  Coverage     78.18%   78.18%           
- Complexity     4041     4042    +1     
=========================================
  Files           428      428           
  Lines         13080    13083    +3     
  Branches       1687     1688    +1     
=========================================
+ Hits          10226    10229    +3     
  Misses         2222     2222           
  Partials        632      632           
Impacted Files Coverage Δ
...rk/runtime/DataVariablesIterationNameProvider.java 100.00% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

codecov[bot] avatar Jun 24 '23 13:06 codecov[bot]

Did you consider to just use a custom test naming for that test? That the data variables are rendered into the iteration name is just the default. But you can customize the default pattern used on all features and you can just use a custom pattern on any individual feature.

Vampire avatar Jun 24 '23 16:06 Vampire

Thank you for checking this PR.

Yes. Currently, I'm using the feature for our product to avoid stacking.

The main reason why I raised this PR is that it was too hard (at least for me) to find out reasons why the tests are not completed. I guess that users need only first few characters of data value name to identify each test cases in almost all cases, and they want to avoid unintentional stacking.

T45K avatar Jun 25 '23 03:06 T45K