dubbo icon indicating copy to clipboard operation
dubbo copied to clipboard

Optimize UT performance by enabling parallel test execution

Open WangzJi opened this issue 5 months ago • 7 comments

  • Add test parallelization configuration in pom.xml
    • Support configurable forkCount and threadCount via properties
    • Enable parallel execution at method level when using multiple forks
    • Default forkCount=1 for backward compatibility
  • Update CI workflow to use FORK_COUNT=2 for parallel execution

What is the purpose of the change?

Closes #15492

Checklist

  • [x] Make sure there is a GitHub_issue field for the change.
  • [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • [x] Write necessary unit-test to verify your logic correction. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
  • [x] Make sure gitHub actions can pass. Why the workflow is failing and how to fix it?

WangzJi avatar Jul 02 '25 08:07 WangzJi

Codecov Report

:x: Patch coverage is 6.66667% with 14 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 37.89%. Comparing base (d879176) to head (18129a3). :warning: Report is 24 commits behind head on 3.3.

:warning: Current head 18129a3 differs from pull request most recent head 246f07d

Please upload reports for the commit 246f07d to get more accurate results.

Files with missing lines Patch % Lines
...n/java/org/apache/dubbo/common/utils/NetUtils.java 6.66% 14 Missing :warning:

:exclamation: There is a different number of reports uploaded between BASE (d879176) and HEAD (18129a3). Click for more details.

HEAD has 4 uploads less than BASE
Flag BASE (d879176) HEAD (18129a3)
unit-tests-java8 1 0
unit-tests-java17 1 0
unit-tests-java11 1 0
unit-tests-java21 1 0
Additional details and impacted files
@@              Coverage Diff              @@
##                3.3   #15521       +/-   ##
=============================================
- Coverage     61.04%   37.89%   -23.15%     
+ Complexity    11702    11689       -13     
=============================================
  Files          1923     1910       -13     
  Lines         87069    86805      -264     
  Branches      13112    13019       -93     
=============================================
- Hits          53152    32897    -20255     
- Misses        28476    49356    +20880     
+ Partials       5441     4552      -889     
Flag Coverage Δ
integration-tests-java21 33.00% <6.66%> (+0.08%) :arrow_up:
integration-tests-java8 33.12% <6.66%> (+0.16%) :arrow_up:
samples-tests-java21 32.69% <6.66%> (+0.09%) :arrow_up:
samples-tests-java8 30.41% <6.66%> (+0.07%) :arrow_up:
unit-tests-java11 ?
unit-tests-java17 ?
unit-tests-java21 ?
unit-tests-java8 ?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov-commenter avatar Jul 02 '25 08:07 codecov-commenter

Some unit tests need to be modified by removing the zookeeper dependency and mocking zookeeper operations, otherwise parallel testing cannot be performed.

zrlw avatar Jul 02 '25 12:07 zrlw

Thanks a lot for your suggestions, really helpful! 🙏 Just to confirm, fixing the current unit test failures should be the right direction for now, right?

WangzJi avatar Jul 02 '25 12:07 WangzJi

Thanks a lot for your suggestions, really helpful! 🙏 Just to confirm, fixing the current unit test failures should be the right direction for now, right?

Right. These unit tests violate unit testing standards and rely on the components that should not be relied upon.

zrlw avatar Jul 02 '25 12:07 zrlw

You can refer to PR #15218, which removed some unit test dependencies on the ZK (dubbo-test) module. Our ultimate goal is to eliminate most of such dependencies on ZK (dubbo-test).

In my opinion, the main reason for the slow unit test execution is that the dubbo-config module uses too many real connections in its unit tests. To address this issue, we should remove those dependencies.

That said, parallelizing the execution of these unit tests, as done in this PR, is also a great improvement.

RainYuY avatar Jul 02 '25 13:07 RainYuY

@Stellar1999 @oxsean @zrlw Thank you for the valuable feedback!

Current Issues Confirmed:

  • CI logs show port conflicts - Tests using fixed ports (2181, 2182) causing failures in parallel execution
  • Too many real ZK connections - dubbo-config module has unnecessary ZooKeeper dependencies
  • Network instability - Embedded ZooKeeper download timeouts in CI environment

Proposed Fix Strategy:

  1. Remove unnecessary ZK dependencies following PR #15218 patterns

    • Replace real ZooKeeper connections with mock addresses
    • Keep only integration tests that truly need ZK functionality
  2. Fix remaining tests to use dynamic ports via NetUtils.getAvailablePort()

I'm thinking most unit tests don't actually need a real ZooKeeper running - they just need to test the code logic. But I want to make sure I understand the boundary correctly.

Looking forward to your guidance! 🙏

WangzJi avatar Jul 04 '25 02:07 WangzJi

@Stellar1999 @oxsean @zrlw Thank you for the valuable feedback!

Current Issues Confirmed:

  • CI logs show port conflicts - Tests using fixed ports (2181, 2182) causing failures in parallel execution
  • Too many real ZK connections - dubbo-config module has unnecessary ZooKeeper dependencies
  • Network instability - Embedded ZooKeeper download timeouts in CI environment

Proposed Fix Strategy:

  1. Remove unnecessary ZK dependencies following PR Remove dubbo-test in dubbo-metrics,dubbo-registry,dubbo-serialization #15218 patterns

    • Replace real ZooKeeper connections with mock addresses
    • Keep only integration tests that truly need ZK functionality
  2. Fix remaining tests to use dynamic ports via NetUtils.getAvailablePort()

I'm thinking most unit tests don't actually need a real ZooKeeper running - they just need to test the code logic. But I want to make sure I understand the boundary correctly.

Looking forward to your guidance! 🙏

Your understanding is correct. I think you can go ahead and fix the second issue first, as this PR is already sufficient as a functional optimization. My previous comment was just to share the evaluation result from my issue. If you're interested in that part(the first), you’re welcome to submit another PR linked to the same issue as this one.

RainYuY avatar Jul 04 '25 02:07 RainYuY