[MSHADE-342] Many integration tests fail when run in isolation

[jira-importer]

Peter De Maeyer opened MSHADE-342 and commented

Maven invoker plugin supports running ITs in isolation, but many ITs fail when run in isolation. One example (there are many others):

mvn -Prun-its clean verify -Dinvoker.test=MSHADE-258_module_relocation

They fail because of a missing target/it/setup-parent/pom.xml. The failure is logged in target/it/setup-parent/build.log.

The reason is that there is a missing parent dependency, which is apparently required for any IT in this project. When you run all the ITs together, it works by accident because the missing directory is created by some other test in the beginning of the IT test run, and then all subsequent tests work as well.

Potential solution

A fix is to identify the broken ITs by running all of them in isolation and add the following to all of the broken ones:

<parent>
    <groupId>org.apache.maven.plugins.shade.its</groupId>
    <artifactId>shade-parent</artifactId>
    <version>1.0</version>
    <relativePath>../setup-parent</relativePath>
</parent>

It should be documented somewhere (maybe it is but I overlooked it?) that, when writing an IT, this must be the parent.

The acceptance criteria for the fix should be:

1. mvn -Run-its clean verify -Dinvoker.test=<any-IT> must run the IT in isolation.
2. Optional, but nice if it works: after running the IT (or all ITs) once, it must be possible to debug the Maven project by doing cd target/it/<any-IT>; mvnDebug install.

(+) It satisfies both acceptance criteria. (-) Every individual IT's pom.xml needs to be updated with a parent dependency it has no business with. (-) It excludes writing an IT for a root project (that has no parent).

Maybe we can find a solution that doesn't involve updating all IT projects. I'll need to investigate a bit more, first I'll need to figure out where the setup-parent project comes from.

---

Affects: 3.2.2

1 votes, 4 watchers

[jira-importer]

Peter De Maeyer commented

It's on my to-do list to fix this, but if someone wants to beat me to it, knock yourself out.

[jira-importer]

Hervé Boutemy commented

nice catch: looks like this IT should be defined as a setup IT = the normal way to show its special role setting relativePath is a good workaround, given the setup is really minimal putting the pom.xml in the root directory and not defining it as setup job could perhaps do even a better job at showing the intent of this parent pom (that is just a config for Java 9+, look also at git blame)

[jira-importer]

Peter De Maeyer commented

this IT should be defined as a setup IT

I don't know how to do that, I'm not so familiar with the maven-invoker-plugin. I'll need to figure that out.

[jira-importer]

Robert Scholte commented

This is a general issue with the maven-invoker-plugin. We should fix it there so other plugins can benefit from it as well.

[jira-importer]

Hervé Boutemy commented

what do you want to fix "here so other plugins can benefit from it as well"? There is a default value for setup ITs https://maven.apache.org/plugins/maven-invoker-plugin/integration-test-mojo.html#setupIncludes , it just has to be followed

notice that in the current case, where it is a basic parent pom, relativePath seems to be less invasive (and if the pom.xml goes in src/it, it's very visible)

[jira-importer]

Hervé Boutemy commented

after writing, I just saw that the convention is followed and works when ITs are not run in isolation (and I'm happy about setup jobs display enhancement from MINVOKER-236 :) )

[INFO] --- maven-invoker-plugin:3.1.0:integration-test (integration-test) @ maven-shade-plugin ---
[INFO] Running 1 setup job:
[INFO] Building: setup-parent/pom.xml
[INFO]           setup-parent/pom.xml ............................. SUCCESS (1.4 s)
[INFO] Setup done.
[INFO] Building: reloc-abs-resource-path-exclude/pom.xml
[INFO]           reloc-abs-resource-path-exclude/pom.xml .......... SUCCESS (3.6 s)

on running setup also on ITs run in isolation, perhaps m-invoker-p can be improved, but I don't see what we can do at m-shade-p level

or once again, since this case is a very basic setup, moving the pom.xml 1 level, using relativePath and dropping the setup IT is IMHO sufficient

[jira-importer]

Mark Struberg commented

I've documented the proper usage over at MASSEMBLY-919. You just have to add that other setup IT as well. e.g.

mvn verify -Dinvoker.test=it-project-parent,projects/bugs/massembly-919 -Prun-its

[jira-importer]

Peter De Maeyer commented

That feels like a workaround and no real solution to me. The downside of that is that you require everyone who wants to run an IT in isolation suddenly needs to "know" about a parent project that the IT project itself has no relation with. Can't we do better than that? The maven-invoker-plugin knows that it needs to run a parent project when you run a test in isolation. I think that the only thing that is missing is that the parent project isn't been copied to the target/ directory, which is causing the failure. Why can't we just fix that and make sure the setup project is copied to the target/ directory? That would be a fix in MINVOKER rather than MSHADE...

[jira-importer]

Hervé Boutemy commented

needs to "know" about a parent project that the IT project itself has no relation with

notice that by convention, these setup jobs are named "setup*", then it should be easier to guess

Why can't we just fix that and make sure the setup project is copied to the target/ directory? a setup job can be complex, it's rarely as simple as the current "setup-parent" job

as a consequence, I don't know if running every setup job when launching an isolated IT is really a good idea... but this discussion is for MINVOKER

FInal point to me is that such a basic parent pom does not require all that IT setup job complexity: if you want a simple solution for the simple current case...

[jira-importer]

Peter De Maeyer commented

I don't know if running every setup job when launching an isolated IT is really a good idea...

I have no prior experience with Maven invoker, but my gut feeling says it would be the right thing to do. My gut feeling is based on the analogy with JUnit 4, with which I have a lot of experience. In JUnit, when you run "all tests", every @BeforeClass is invoked once before all tests in a test class and @AfterClass is invoked after. When running an isolated test in a class, e.g. from the right-click menu in your IDE, the @BeforeClass and @AfterClass methods are also run before and after. It is elegant and corresponds to intuition, it seems like the right thing to do.

Indeed it would be best to move this issue to MINVOKER.