changing depedencies and build plan to be able to use parallel buld or mvnd and to be sure artifacts are available when running its use verifier in auto mode (e.g embedded most of the time) or mvnd and use verifier in auto mode (e.g embedded most of the time)

[olamy]

trafficstars

• try to include those dependencies to fix the build plan and be sure its are runned last when using -Txx or mvnd
• use verifier in auto mode to have its running faster

Following this checklist to help us incorporate your contribution quickly and easily:

• [ ] Make sure there is a JIRA issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes.
• [ ] Each commit in the pull request should have a meaningful subject line and body.
• [ ] Format the pull request title like [SUREFIRE-XXX] - Fixes bug in ApproximateQuantiles, where you replace SUREFIRE-XXX with the appropriate JIRA issue. Best practice is to use the JIRA issue title in the pull request title and in the first line of the commit message.
• [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
• [ ] Run mvn clean install to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
• [ ] You have run the integration tests successfully (mvn -Prun-its clean install).

If your pull request is about ~20 lines of code you don't need to sign an Individual Contributor License Agreement if you are unsure please ask on the developers list.

To make clear that you license your contribution under the Apache License Version 2.0, January 2004 you have to acknowledge this by using the following check-box.

• [ ] I hereby declare this contribution to be licenced under the Apache License Version 2.0, January 2004

• [ ] In any other case, please file an Apache Individual Contributor License Agreement.

[Tibor17]

@olamy This does not make sense. First of all the Maven should have very stable Resolver avoiding file locking in the Maven local repo. I used ParallelParameterized tests, I can do it again, but due to the Maven 3.2.x locks artifacts, ParallelParameterized won't be used.

[olamy]

@olamy This does not make sense. First of all the Maven should have very stable Resolver avoiding file locking in the Maven local repo. I used ParallelParameterized tests, I can do it again, but due to the Maven 3.2.x locks artifacts, ParallelParameterized won't be used.

Dear @Tibor17 There is nothing to do with Resolver or whatever. As explained in the description the goal is to be able to use mvn -Tx or mvnd to have a faster build (some modules can definitely be build in parallel). But doing that surefire-its is build from the start as it doesn't have any dependencies declared to the modules it will test! Which definitely doesn't reflect what the module needs to be build first. As far I can tell the surefire-its test modules which test the maven-surefire-plugin and maven-failsafe-plugin definitely depends on the plugins to be build first.

[slawekjaranowski]

related issue https://issues.apache.org/jira/browse/SUREFIRE-1956

[olamy]

related issue https://issues.apache.org/jira/browse/SUREFIRE-1956

definitely related

@Tibor17

just run mvn clean verify -pl :surefire-its -am and you will see dependencies are not declared correctly

with master

➜  maven-surefire git:(master) mvn clean verify -pl :surefire-its -am
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Apache Maven Surefire                                              [pom]
[INFO] Maven Surefire Integration Tests                                   [jar]
[INFO] 
[INFO] -----------------< org.apache.maven.surefire:surefire >-----------------
[INFO] Building Apache Maven Surefire 3.0.0-M6-SNAPSHOT                   [1/2]
[INFO] --------------------------------[ pom ]---------------------------------

with this branch:

➜  maven-surefire git:(multithread-build-possible) mvn clean verify -pl :surefire-its -am                    
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Apache Maven Surefire                                              [pom]
[INFO] Surefire Shared Utils                                              [jar]
[INFO] SureFire Logger API                                                [jar]
[INFO] SureFire API                                                       [jar]
[INFO] Surefire Extensions API                                            [jar]
[INFO] Surefire Extensions SPI                                            [jar]
[INFO] SureFire Booter                                                    [jar]
[INFO] Maven Surefire Common                                              [jar]
[INFO] Surefire Report Parser                                             [jar]
[INFO] Maven Surefire Plugin                                     [maven-plugin]
[INFO] Maven Failsafe Plugin                                     [maven-plugin]
[INFO] Maven Surefire Report Plugin                              [maven-plugin]
[INFO] Maven Surefire Integration Tests                                   [jar]

it is more correct but still wrong as this should include everything.

[slawekjaranowski]

From other side we don't have many boost by building project modules by many thread, at the end we land in surefire-its which consumes all time.

I thinking to use Verifier in embedded mode for it tests like in Maven core its.

[olamy]

From other side we don't have many boost by building project modules by many thread, at the end we land in surefire-its which consumes all time.

I thinking to use Verifier in embedded mode for it tests like in Maven core its.

agree but it's still something to fix as currently you could run its without running the rest of dependencies first.

[Tibor17]

When I see the commit, I have to say that this PR is a pure mess and disaster, the providers must not be touched in the ITs POM. The POM changes have nothing to do with any build speed up, it has nothing to do with -T3. It's a pure hack which affects functionality of the tests and it is very risky solution. As I said before in Slack or GH, the way how the build can be faster is to focus on the most slow IT classes and those are only few and those are Parameterized and they can be speed up as it was before (I deleted due to Maven Resolver) only the way that the Maven local repo would have all artifacts pre-loaded or the Resolver would be failsafe which is not now and so the concurrent ITs would fail the Resolver of course. Maiking the project build parallel would speed up the build in 5 min because all modules except the surefire-its are much faster in comparison to the ITs, so there is no reason to think about these "improvements". The penalty would wrong behavior of the tests, and that's worst!

Another story is embedded Maven. This was not stable solution for us some time ago because it shares system props and env is shared and the developer has no notion about while writing the test, and so there is the risc the IT however succeeds on Jenkins but fails in real environment. That's the reason why it was removed several years ago in the surefire-its/pom.xml.

[Tibor17]

If somevody wants to keep the build order of some groupd of modules, and some other group of modules should be built in parallel, then hacking is not in place in our Apache project, that we hack ourselves. In this case you should propose a feature in Maven Core, as for instance this:

<modules concurrency="preferGivenOrder">
  <module>a</modules>
  <module parallelGroup="b">b1</modules>
  <module parallelGroup="c">c1</modules>
  <module parallelGroup="b">b2</modules>
  <module parallelGroup="c">c2</modules>
  <module>d</modules>
</modules>

[olamy]

I agree hacking was a bad choice of title but it was just a quick one when first looking at it. Maybe fixing would have been better but I updated the title to changing.

[olamy]

When I see the commit, I have to say that this PR is a pure mess and disaster, the providers must not be touched in the ITs POM. The POM changes have nothing to do with any build speed up, it has nothing to do with -T3. It's a pure hack which affects functionality of the tests and it is very risky solution. As I said before in Slack or GH, the way how the build can be faster is to focus on the most slow IT classes and those are only few and those are Parameterized and they can be speed up as it was before (I deleted due to Maven Resolver) only the way that the Maven local repo would have all artifacts pre-loaded or the Resolver would be failsafe which is not now and so the concurrent ITs would fail the Resolver of course. Maiking the project build parallel would speed up the build in 5 min because all modules except the surefire-its are much faster in comparison to the ITs, so there is no reason to think about these "improvements". The penalty would wrong behavior of the tests, and that's worst!

Another story is embedded Maven. This was not stable solution for us some time ago because it shares system props and env is shared and the developer has no notion about while writing the test, and so there is the risc the IT however succeeds on Jenkins but fails in real environment. That's the reason why it was removed several years ago in the surefire-its/pom.xml.

2 facts:

1. build plan

• before this commits in master:

➜  maven-surefire git:(master) mvn clean install -pl :surefire-its -am
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Apache Maven Surefire                                              [pom]
[INFO] Maven Surefire Integration Tests                                   [jar]

• after this change:

➜  maven-surefire git:(multithread-build-possible) mvn clean install -pl :surefire-its -am
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Apache Maven Surefire                                              [pom]
[INFO] Surefire Shared Utils                                              [jar]
[INFO] SureFire Logger API                                                [jar]
[INFO] SureFire API                                                       [jar]
[INFO] Surefire Extensions API                                            [jar]
[INFO] Surefire Extensions SPI                                            [jar]
[INFO] SureFire Booter                                                    [jar]
[INFO] Maven Surefire Test-Grouping Support                               [jar]
[INFO] SureFire Providers                                                 [pom]
[INFO] Shared JUnit3 Provider Code                                        [jar]
[INFO] Shared Java 5 Provider Base                                        [jar]
[INFO] Shared JUnit4 Provider Code                                        [jar]
[INFO] Shared JUnit48 Provider Code                                       [jar]
[INFO] SureFire JUnit Runner                                              [jar]
[INFO] SureFire JUnit4 Runner                                             [jar]
[INFO] Maven Surefire Common                                              [jar]
[INFO] SureFire JUnitCore Runner                                          [jar]
[INFO] SureFire JUnit Platform Runner                                     [jar]
[INFO] SureFire TestNG Utils                                              [jar]
[INFO] SureFire TestNG Runner                                             [jar]
[INFO] Surefire Report Parser                                             [jar]
[INFO] Maven Surefire Plugin                                     [maven-plugin]
[INFO] Maven Failsafe Plugin                                     [maven-plugin]
[INFO] Maven Surefire Report Plugin                              [maven-plugin]
[INFO] Maven Surefire Integration Tests                                   [jar]

I tend to think the second build plan is more correct regarding to what the surefire-its wants to achieve. the goal is not really to use only parallel as it will provide only a bit shorter build time but hey it's still an improvement.

2. use of verifier with auto mode:

• without this any branch of the build is about 1h20minutes see https://ci-maven.apache.org/job/Maven/job/maven-box/job/maven-surefire/job/master/
• with this change: https://ci-maven.apache.org/blue/organizations/jenkins/Maven%2Fmaven-box%2Fmaven-surefire/detail/PR-479/33/pipeline/24 branch are about 50 minutes (still issues to have a look for jdk11+) (locally the build is down from 1h10m to 42minutes Personally I think it worth it just to help us developing and contributor to not waste too much time.

[olamy]

grhhhhh those ITs tests are so flaky depending on external resources 😩

[2022-07-28T07:38:55.790Z] Caused by: org.apache.maven.wagon.TransferFailedException: transfer failed for https://repo.maven.apache.org/maven2/org/apache/maven/wagon/wagon-provider-api/3.3.1/wagon-provider-api-3.3.1.jar

[2022-07-28T07:38:55.790Z]     at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1250)

[2022-07-28T07:38:55.790Z]     at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1140)

[2022-07-28T07:38:55.790Z]     at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126)

[2022-07-28T07:38:55.790Z]     at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88)

[2022-07-28T07:38:55.790Z]     at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61)