cucumber-jvm
cucumber-jvm copied to clipboard
Reduced mockito usage
🤔 What's changed?
I reduced the usage of the Mockito framework. This is a first pass, focused on the simple cases (typically: replacing the mock+when constructs with stub classes).
⚡️ What's your motivation?
Fixes #2707 .
By focusing on the simple cases we have a big pull request of stuff that is simple to review. The other replacements are more complex and prone to more discussions, so I will make another pull request for them.
🏷️ What kind of change is this?
- :bank: Refactoring/debt/DX (improvement to code design, tooling, documentation etc. without changing behaviour)
♻️ Anything particular you want feedback on?
Performance when running all tests using Intellij (right click on project -> Run -> All Tests ; average duration over 5 runs):
project | test duration (before -> after) [s] | mockito contribution (before -> after) |
---|---|---|
cucumber-junit | 3.3 -> 1.1 | 2.2 s = 68% -> 0% (removed dependency) |
cucumber-core | 9.3 -> 6.4 | 2.9 s = 31% -> 0% (removed dependency) |
cucumber-java | 2.7 -> 0.8 | 1.9 s = 72% -> 0% (removed dependency) |
cucumber-java8 | 0.9 -> 0.7 | 0.2 s = 21% -> 0% (removed dependency) |
cucumber-spring | 5.3 -> 3.2 | 2.0 s = 39% -> 0% (removed dependency) |
cucumber-testng | 1.7 -> 1.7 | 0.0 s = 0% -> 0% (removed dependency) |
cucumber-guice | 2.4 -> 0.7 | 1.7 s = 71% -> 0% (removed dependency) |
datatable | 0.5 -> 0.5 | (unused dependency) 0.0 s = 0% -> 0% (removed dependency) |
Thus, when running tests interactively, we see a good improvement by not using mockito (up to 72% of testing time on cucumber-java).
~However, over the full build (maven clean install on cucumber-jvm), there is no build time difference: on my machine, the build time is 1 minute 49 seconds on the main
branch as well as on branch reduce-mockito-usage-pass1
(average build time over 5 builds).~
~My guess is that the build time is much longer than the test duration, so the test duration on total maven clean+install is not noticeable.~
Edit:
One year after, I did additional tests on my local machine for a maven clean install on cucumber-jvm
(5 times average):
- with mockito (
main
branch ): average=3 minutes 42 seconds (standard deviation = 8 seconds) - without mockito (
reduce-mockito-usage-pass1
branch): average=3 minutes 26 seconds (standard deviation = 12 seconds)
That is: the build is 7% faster without mockito.
📋 Checklist:
- [x] I agree to respect and uphold the Cucumber Community Code of Conduct
- [ ] I've changed the behaviour of the code
- [ ] I have added/updated tests to cover my changes.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [ ] Users should know about my change
- [ ] I have added an entry to the "Unreleased" section of the CHANGELOG, linking to this pull request.