Using a file-based P2 Repository in 'repository' section in the POM increases build time

[sparsick]

Tycho Version: 2.3.0 Maven Version: 3.6.3 Java Version: 11

I change the http-based P2 repository to a file-based P2 repository in the repository section in the POM

        <repository>
          <id>main</id>
          <url>file:${basedir}/../target-platform</url>
          <layout>p2</layout>
        </repository>

This change increases the build time, although I would expect a faster build time.

The reason is the following log output (only a snippet):

[INFO] Reading com.fasterxml.jackson.jaxrs.jackson-jaxrs-base.source_2.5.4.jar
[INFO] Reading com.fasterxml.jackson.jaxrs.jackson-jaxrs-base_2.5.4.jar
[INFO] Reading com.fasterxml.jackson.jaxrs.jackson-jaxrs-json-provider.source_2.5.4.jar
[INFO] Reading com.fasterxml.jackson.jaxrs.jackson-jaxrs-json-provider_2.5.4.jar
[INFO] Reading com.fasterxml.jackson.module.jackson-module-jaxb-annotations.source_2.5.4.jar
[INFO] Reading com.fasterxml.jackson.module.jackson-module-jaxb-annotations_2.5.4.jar
[INFO] Reading javassist.source_3.18.1.GA.jar
[INFO] Reading javassist_3.18.1.GA.jar
[INFO] Reading javax.annotation-api.source_1.2.0.jar
[INFO] Reading javax.annotation-api_1.2.0.jar
[INFO] Reading javax.validation.api.source_1.1.0.Final.jar
[INFO] Reading javax.validation.api_1.1.0.Final.jar
[INFO] Reading javax.ws.rs-api.source_2.0.1.jar
[INFO] Reading javax.ws.rs-api_2.0.1.jar
[INFO] Reading org.glassfish.hk2.api.source_2.4.0.b34.jar
[INFO] Reading org.glassfish.hk2.api_2.4.0.b34.jar
[INFO] Reading org.glassfish.hk2.external.aopalliance-repackaged.source_2.4.0.b34.jar
[INFO] Reading org.glassfish.hk2.external.aopalliance-repackaged_2.4.0.b34.jar
[INFO] Reading org.glassfish.hk2.external.javax.inject.source_2.4.0.b34.jar
[INFO] Reading org.glassfish.hk2.external.javax.inject_2.4.0.b34.jar
[INFO] Reading org.glassfish.hk2.locator.source_2.4.0.b34.jar
[INFO] Reading org.glassfish.hk2.locator_2.4.0.b34.jar
[INFO] Reading org.glassfish.hk2.osgi-resource-locator.source_1.0.1.jar
[INFO] Reading org.glassfish.hk2.osgi-resource-locator_1.0.1.jar
[INFO] Reading org.glassfish.hk2.utils.source_2.4.0.b34.jar
[INFO] Reading org.glassfish.hk2.utils_2.4.0.b34.jar
[INFO] Reading org.glassfish.jersey.bundles.repackaged.jersey-guava.source_2.23.0.jar
[INFO] Reading org.glassfish.jersey.bundles.repackaged.jersey-guava_2.23.0.jar
[INFO] Reading org.glassfish.jersey.core.jersey-client.source_2.23.0.jar
[INFO] Reading org.glassfish.jersey.core.jersey-client_2.23.0.jar
[INFO] Reading org.glassfish.jersey.core.jersey-common.source_2.23.0.jar
[INFO] Reading org.glassfish.jersey.core.jersey-common_2.23.0.jar
[INFO] Reading org.glassfish.jersey.core.jersey-server.source_2.23.0.jar
[INFO] Reading org.glassfish.jersey.core.jersey-server_2.23.0.jar
[INFO] Reading org.glassfish.jersey.ext.jersey-entity-filtering.source_2.23.0.jar
[INFO] Reading org.glassfish.jersey.ext.jersey-entity-filtering_2.23.0.jar
[INFO] Reading org.glassfish.jersey.media.jersey-media-jaxb.source_2.23.0.jar
[INFO] Reading org.glassfish.jersey.media.jersey-media-jaxb_2.23.0.jar
[INFO] Reading org.glassfish.jersey.media.jersey-media-json-jackson.source_2.23

Calling the build with mvn install | grep -vE 'Reading' to avoid the above output decreases the build time significantly.

This code section is responsible for the "Reading" output.

It seems, for me, that the "Reading" output is only needed for debugging, so my proposal is to move this log output from log level INFO to DEBUG. If you agree with this assessment, I'd like to create a PR for that.

[mickaelistria]

I'm not sure the logging itself is the issue; the FileTargetDefinitionContent seems to perform a lot of extra work which can cause slowness. As this is a legal URI of a regular p2 repo, the URITargetDefinitionContent should be used instead for that case, this is usually much faster. There is most likely something in the code that takes the wrong decision for this case, leading to usage of FileTargetDefinitionContent instead of URITargetDefinitionContent .

[laeubi]

I must confess I can't believe that

change the http-based P2 repository to a file-based P2 repository

is the cause:

1. FileTargetDefinitionContent is only used when resolving a target-file but here a repository reference in the pom is given
2. Even if it would be, FileTargetDefinitionContent requires a File, while here an URI (with file scheme(!)) is used and we never convert URI->File, nor do we change from a InstallableUnit-type location to a directory-type location
3. @mickaelistria even mentioned here that repository sections currently do not support target locations, so how should this <repository> -> target -> InstallableUnit -> Directory conversation happen?
4. FileTargetDefinitionContent has only one reference in the whole tycho source, so it is very unlikely that something happens unintentionally

I would more suspect that

change the http-based P2 repository to a file-based P2 repository

was not the only change but that ${basedir}/../target-platform actually contains a target file that has a Directory type entry (probably for local development) that points to the p2-repository and this is then resolved as desired using the plugins/features folder that one can also find in a P2 site folder layout.

So even

it seems, for me, that the "Reading" output is only needed for debugging,

is not completely true as it reveals a configuration error here, but I'm totally open to change this to DEBUG in any case, as strictly speaking no output is required for the actual functionality of any maven feature :-)

[mickaelistria]

@sparsick Can you please provide a very minimal reproducer and enable debug output?

[sparsick]

@laeubi Thanks for your explanation. Our RCP project has 50 modules, so it is absolutely possible that I overlooked some target definition files or other repository settings. I will recheck it. I was confused because after my above changes, the Reading output appears the first time in the output. Therefore, my assumption the FileTargetDefinitionContent could be responsible for that.

@mickaelistria I will try to set up a minimal reproducer for this problem.

[laeubi]

@sparsick is this still an issue for you?

[Bananeweizen]

@sparsick Be aware the documentation meanwhile has some profiling advice: https://github.com/eclipse/tycho/blob/master/CONTRIBUTING.md#profiling-the-tycho-build. Especially with YouMonitor you can compare your company project build times with and without that change, without trying to make a reproducer publicly available.

[sparsick]

@laeubi I had the last six month some health issues, so I had no time for this issue :( . Sorry for that. I started working some weeks ago, partly. I hope that I can invest time in this issue in the next weeks.

@Bananeweizen Thank you for your hint.

[laeubi]

@sparsick As no one else had reported the issue and we have no reproducer for this I'll close the issue for now, but feel free to ask to reopen it if you find time to further investigate.