cucumber-jvm icon indicating copy to clipboard operation
cucumber-jvm copied to clipboard

Reduced mockito usage

Open jkronegg opened this issue 1 year ago • 3 comments

🤔 What's changed?

I reduced the usage of the Mockito framework. This is a first pass, focused on the simple cases (typically: replacing the mock+when constructs with stub classes).

⚡️ What's your motivation?

Fixes #2707 .

By focusing on the simple cases we have a big pull request of stuff that is simple to review. The other replacements are more complex and prone to more discussions, so I will make another pull request for them.

🏷️ What kind of change is this?

  • :bank: Refactoring/debt/DX (improvement to code design, tooling, documentation etc. without changing behaviour)

♻️ Anything particular you want feedback on?

Performance when running all tests using Intellij (right click on project -> Run -> All Tests ; average duration over 5 runs):

project test duration (before -> after) [s] mockito contribution (before -> after)
cucumber-junit 3.3 -> 1.1 2.2 s = 68% -> 0% (removed dependency)
cucumber-core 9.3 -> 6.4 2.9 s = 31% -> 0% (removed dependency)
cucumber-java 2.7 -> 0.8 1.9 s = 72% -> 0% (removed dependency)
cucumber-java8 0.9 -> 0.7 0.2 s = 21% -> 0% (removed dependency)
cucumber-spring 5.3 -> 3.2 2.0 s = 39% -> 0% (removed dependency)
cucumber-testng 1.7 -> 1.7 0.0 s = 0% -> 0% (removed dependency)
cucumber-guice 2.4 -> 0.7 1.7 s = 71% -> 0% (removed dependency)
datatable 0.5 -> 0.5 (unused dependency) 0.0 s = 0% -> 0% (removed dependency)

Thus, when running tests interactively, we see a good improvement by not using mockito (up to 72% of testing time on cucumber-java).

~However, over the full build (maven clean install on cucumber-jvm), there is no build time difference: on my machine, the build time is 1 minute 49 seconds on the main branch as well as on branch reduce-mockito-usage-pass1 (average build time over 5 builds).~

~My guess is that the build time is much longer than the test duration, so the test duration on total maven clean+install is not noticeable.~

Edit: One year after, I did additional tests on my local machine for a maven clean install on cucumber-jvm (5 times average):

  • with mockito ( main branch ): average=3 minutes 42 seconds (standard deviation = 8 seconds)
  • without mockito (reduce-mockito-usage-pass1 branch): average=3 minutes 26 seconds (standard deviation = 12 seconds)

That is: the build is 7% faster without mockito.

📋 Checklist:

  • [x] I agree to respect and uphold the Cucumber Community Code of Conduct
  • [ ] I've changed the behaviour of the code
    • [ ] I have added/updated tests to cover my changes.
  • [ ] My change requires a change to the documentation.
    • [ ] I have updated the documentation accordingly.
  • [ ] Users should know about my change
    • [ ] I have added an entry to the "Unreleased" section of the CHANGELOG, linking to this pull request.

jkronegg avatar Jun 06 '23 05:06 jkronegg