Bug: 1052 Adding integration test for Mirror can't find product bundle dependencies.

[fipro78]

[github-actions[bot]]

Unit Test Results

270 tests   265 :heavy_check_mark:  48m 2s :stopwatch: 157 suites      4 :zzz: 157 files        0 :x:  1 :fire:

For more details on these errors, see this check.

Results for commit dfca2f3b.

[github-actions[bot]]

Test Results

  582 files  +3    582 suites  +3   4h 1m 43s :stopwatch: + 36m 28s   385 tests +1    375 :white_check_mark:  - 2   7 :zzz: ±0  2 :x: +2  1 :fire: +1  1 155 runs  +3  1 128 :white_check_mark:  - 2  22 :zzz: ±0  2 :x: +2  3 :fire: +3 

For more details on these failures and errors, see this check.

Results for commit 6a3e4acf. ± Comparison against base commit 6b9cefeb.

:recycle: This comment has been updated with latest results.

[laeubi]

@fipro78 can you try to minimize the testcase as indicated in the bug report? I think a smaller test-case can help us to identify the root cause more quickly.

[fipro78]

@laeubi well I can try to if I remember what I have done back then. But IIRC you also asked for a testcase about plugin based products. Shouldn't then there be plugins that are integrated? IIUC you want me to drop everything else but the product and the target right?

[laeubi]

Yes I think you can drop all the reactor projects (beside target+product of course), and even reduce the product to only contain the org.osgi.service.jaxrs bundle as a reproducer. Then one even can reduce the target further (smaller target = faster execution times) so we get the smallest possible reproducing test-case :-)

This should then make debugging a lot easier as we have only a handful of bundles in the debug in contrast to now a few hundreds.

[fipro78]

I removed everything and tried to build the project alone locally, but it fails with this error:

[WARNING] The POM for org.eclipse.tycho:tycho-build:jar:3.0.0-SNAPSHOT is missing, no dependency information available
[WARNING] Failed to read extensions descriptor C:\Development\Sources\tycho\tycho-its\projects\product.pluginBased.mavenArtefacts\.mvn\extensions.xml: Plugin org.eclipse.tycho:tycho-build:3.0.0-SNAPSHOT or one of its dependencies could not be resolved: Could not find artifact org.eclipse.tycho:tycho-build:jar:3.0.0-SNAPSHOT
[INFO] Scanning for projects...
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[ERROR] Child module C:\Development\Sources\tycho\tycho-its\projects\product.pluginBased.mavenArtefacts\tycho-its-project.issue1052.target\pom.xml of C:\Development\Sources\tycho\tycho-its\projects\product.pluginBased.mavenArtefacts\pom.xml does not exist @
[ERROR] Child module C:\Development\Sources\tycho\tycho-its\projects\product.pluginBased.mavenArtefacts\tycho-its-project.issue1052.jaxrs.product\pom.xml of C:\Development\Sources\tycho\tycho-its\projects\product.pluginBased.mavenArtefacts\pom.xml does not exist @
[ERROR] Unresolveable build extension: Plugin org.eclipse.tycho:tycho-maven-plugin:3.0.0-SNAPSHOT or one of its dependencies could not be resolved: Failed to collect dependencies at org.eclipse.tycho:tycho-maven-plugin:jar:3.0.0-SNAPSHOT -> org.eclipse.tycho:tycho-core:jar:3.0.0-SNAPSHOT -> org.eclipse.platform:org.eclipse.core.runtime:jar:3.25.0 -> org.eclipse.platform:org.eclipse.equinox.preferences:jar:[3.10.0,4.0.0) @

No idea what is wrong. The pom.xml contains the plugins-repository and I trigger the build via

mvn -Dtycho-version=3.0.0-SNAPSHOT clean verify

Any ideas?

[laeubi]

Are you using maven 3.8.6?

[laeubi]

Sorry for confusion, I think this is because the snapshot repository needs to be configured in the settings.xml

https://github.com/eclipse-tycho/tycho/wiki#configure-snapshots-repository-globally

[fipro78]

@laeubi I have updated the test case. It shows several errors related to some signing topics, but at the end it fails with the reported error.

[laeubi]

@fipro78 I have now debugged this and the problem is that there are Projector warnings that prevent a solution here:

Status WARNING: org.eclipse.equinox.p2.director code=0 Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; javax.ws.rs.client 0.0.0.
Status WARNING: org.eclipse.equinox.p2.director code=0 Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; javax.ws.rs.core 0.0.0.
Status WARNING: org.eclipse.equinox.p2.director code=0 Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; javax.ws.rs.sse 0.0.0.
Status WARNING: org.eclipse.equinox.p2.director code=0 Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; org.osgi.dto [1.0.0,2.0.0).
Status WARNING: org.eclipse.equinox.p2.director code=0 Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; org.osgi.framework.dto [1.8.0,2.0.0).
Status WARNING: org.eclipse.equinox.p2.director code=0 Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to osgi.contract; (&(osgi.contract=JavaJAXRS)(version=2.1.0)).

This then leads to the situation (as all requirements of a product are considered "non greedy" and optional) that the final result of the target only contains [a.jre.javase 17.0.0, tycho-its-project.issue1052.jaxrs.product 1.0.0.qualifier] what of course is not enough. It then (later on) fails to publish the content to the repository that is more later used to perform the mirror operation.

[laeubi]

@fipro78 can you tell if you would have expsected that the reported requirements are already fulfilled? If yes by what bundle(s) from the target platform?

[fipro78]

@laeubi Well initially the product was working. The target and the product contained everything that was needed and it was possible to start the product via PDE inside Eclipse. You then asked me to drop everything to make the example simpler for debugging. So I really can't tell as with this modification the example got broken. Probably I removed to much.

[laeubi]

What I suspect here is that the product would actually works, but P2 actually has a problem here because it can't understand some of the requirements. At least the actual error seems misleading here.

[fipro78]

IIRC the initial product configuration did not show theses warnings, e.g. org.osgi.dto is available if Equinox is included. And I think I also added the necessary xml libraries. I would need to restore the initial setting to see. IMHO the error you see now is different than the initial one.

[laeubi]

IIRC the initial product configuration did not show theses warnings,

Well that is because until now these warnings are never printed ;-)

I added some output with this PR:

• https://github.com/eclipse-tycho/tycho/pull/1416

[laeubi]

By the way that are the units that are part of the current target platform: tycho-its-project.issue1052.jaxrs.product 1.0.0.qualifier osgi.annotation.source 8.0.1.202109301733 org.osgi.namespace.service.source 1.0.0.201505202024 org.osgi.util.promise.source 1.1.1.201810101357 org.osgi.namespace.implementation.source 1.0.0.201505202024 osgi.annotation 8.0.1.202109301733 org.osgi.service.jaxrs 1.0.1.202109301733 org.osgi.service.jaxrs.source 1.0.1.202109301733 org.osgi.util.promise 1.1.1.201810101357 org.osgi.namespace.service 1.0.0.201505202024 org.osgi.util.function 1.1.0.201802012106 org.osgi.util.function.source 1.1.0.201802012106 org.osgi.namespace.implementation 1.0.0.201505202024 a.jre.javase 17.0.0

[fipro78]

The initial target definition contained all necessary dependencies. You can see this in the first commit. While the service api and impl bundles are not necessary for the test, it seems the rest is necessary to show the real error. That said, the initial commit should have been only reduced by removing the api and impl, not the rest.

[laeubi]

it seems the rest is necessary to show the real error. That said, the initial commit should have been only reduced by removing the api and impl, not the rest.

I then get the following warnings from the projector:

Unable to satisfy dependency from org.apache.aries.jax.rs.whiteboard 1.0.1 to osgi.contract; (&(osgi.contract=JavaJAXRS)(version=2.1.0)).
Unable to satisfy dependency from org.apache.aries.jax.rs.whiteboard 1.0.1 to osgi.contract; (&(osgi.contract=JavaAnnotation)(version=1.3.0)).
Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.0.201802012106 to osgi.contract; (&(osgi.contract=JavaJAXRS)(version=2.1.0)).
Unable to satisfy dependency from slf4j.api 1.7.2 to java.package; org.slf4j.impl 1.6.0.

The list of resolved units is:

org.osgi.service.jaxrs 1.0.0.201802012106
jakarta.xml.bind-api 2.3.3
jakarta.annotation-api 1.3.5
org.apache.felix.gogo.runtime 1.1.4.v20210111-1007
org.apache.servicemix.specs.annotation-api-1.3 1.3.0.1
org.osgi.util.promise 1.1.0.201802012106
com.sun.activation.javax.activation 1.2.0
org.eclipse.equinox.executable_root.win32.win32.x86_64 3.8.1700.v20220509-0833
org.eclipse.pde.ds.lib 1.1.500.v20210209-1250
org.slf4j.api 1.7.30.v20200204-2150
javax.annotation 1.3.5.v20200909-1856
org.eclipse.equinox.executable.feature.jar 3.8.1700.v20220509-0833
org.eclipse.osgi 3.18.0.v20220516-2155
a.jre.javase 17.0.0
org.eclipse.equinox.http.servlet 1.7.200.v20211021-1418
org.osgi.util.function 1.1.0.201802012106
org.osgi.util.function 1.2.0.202109301733
org.osgi.service.http.whiteboard 1.0.0.201505202024
slf4j.api 1.7.2
org.eclipse.equinox.launcher.win32.win32.x86_64 1.2.500.v20220509-0833
org.eclipse.osgi.services 3.10.200.v20210723-0643
org.apache.servicemix.specs.jaxws-api-2.2 2.9.0
org.eclipse.equinox.launcher 1.6.400.v20210924-0641
org.apache.felix.gogo.command 1.1.2.v20210111-1007
org.apache.felix.scr 2.1.24.v20200924-1939
org.osgi.util.promise 1.2.0.202109301733
org.apache.felix.gogo.shell 1.1.4.v20210111-1007
org.eclipse.equinox.executable.feature.group 3.8.1700.v20220509-0833
org.apache.aries.jax.rs.whiteboard 1.0.1
org.eclipse.equinox.console 1.4.500.v20211021-1418
org.eclipse.equinox.supplement 1.10.500.v20220516-2155
jakarta.servlet-api 4.0.0
tycho-its-project.issue1052.jaxrs.product 1.0.0.qualifier
org.apache.felix.http.servlet-api 1.1.2
org.apache.aries.javax.jax.rs-api 1.0.1
org.apache.servicemix.specs.saaj-api-1.3 2.9.0
tycho-its-project.issue1052.impl 1.0.0.202209251633
org.apache.felix.http.jetty 4.1.14
tycho-its-project.issue1052.api 1.0.0.202209251633

[fipro78]

I would think that the contracts should be resolved via

org.apache.aries.javax.jax.rs-api org.apache.servicemix.specs.annotation-api

I wonder why the annotation api is resolved via three different jars. Also SLF4J api is available twice. The resolution error does not make sense to me.

[laeubi]

@merks are you aware of any problems regarding P2 and osgi.contract handling?

[laeubi]

I would think that the contracts should be resolved via

org.apache.aries.javax.jax.rs-api org.apache.servicemix.specs.annotation-api

If I look at this I see

Provide-Capability: osgi.contract;osgi.contract=JavaJAXRS;uses:="javax
 .ws.rs,javax.ws.rs.client,javax.ws.rs.container,javax.ws.rs.core,java
 x.ws.rs.ext,javax.ws.rs.sse";version:List<Version>="2.1,2,1.1"

as there is a version LIST @HannesWell I remember you have investigated some missing support in P2 regarding list support?

Could it be:

• https://github.com/eclipse-equinox/p2/pull/64 ?

[HannesWell]

I would think that the contracts should be resolved via org.apache.aries.javax.jax.rs-api org.apache.servicemix.specs.annotation-api

If I look at this I see

Provide-Capability: osgi.contract;osgi.contract=JavaJAXRS;uses:="javax
 .ws.rs,javax.ws.rs.client,ja

as there is a version **LIST** @HannesWell I remember you have investigated some missing support in P2 regarding list support?

Could it be:

* [Fix creation of generic-capabilities with multiple versions eclipse-equinox/p2#64](https://github.com/eclipse-equinox/p2/pull/64) ?

Yes that's right and it could definitivly be connected. But a question that I asked myself a few times already but never answered is, does Tycho/P2 even consider Require-/Provide-Capability Headers in general?

[laeubi]

But a question that I asked myself a few times already but never answered is, does Tycho/P2 even consider Require-/Provide-Capability Headers in general?

Obviously it does or we wont see this error here, but maybe I don't fully understand the question?

[merks]

Here's a simple way to see what the p2 publisher does. Using `platform:/resource/ in the Repository Explorer shows the p2 published results of all workspace projects. Turning on Expert mode lists all the IUs. You can filter to find one and you can double click the version to see the serialized-to-XML p2 metadata of the IU.

There should be nothing special about how osgi.contract is handled...

[laeubi]

@fipro78 Tycho is updated to new eclipse release so with some small adjustments this should finally work now!

[laeubi]

@fipro78 I know this is a very long outstanding issue so sorry for the delay, but I think the root cause might be this here:

• https://github.com/eclipse-tycho/tycho/pull/3389

lets see if it not breaks anything and after the merge I hope this will be finally resolved. If not I'm a bit out of clue as it seems no one else is really using plugin based products or facing the issue ...

[laeubi]

@fipro78 I now have analyzed the problem and actually it works, but the failure we see here is introduced by some mismatch between maven metadata and OSGi metadata:

1. https://mvnrepository.com/artifact/org.osgi/org.osgi.service.jaxrs/1.0.1 declares the org.apache.aries.javax.jax.rs-api as a provided dependency, that means it is not included in the target.
2. but org.osgi.service.jaxrs require package javax.ws.rs.client
3. If one adds this, dependency it reveals that org.apache.aries.javax.jax.rs-api does not declare any maven dependencies but it requires package javax.xml.bind.annotation
4. If one now adds for example https://mvnrepository.com/artifact/javax.xml.bind/jaxb-api/2.3.1 and what requires https://mvnrepository.com/artifact/javax.activation/javax.activation-api/1.2.0 then the build passes.

Another issue is that Tycho does not fail when resolve the product, that's a bit strange....

[laeubi]

I now replicated the behavior with two minimal test cases:

• https://github.com/eclipse-tycho/tycho/pull/3393

this shows:

1. feature based product fails to resolve with the error that package javax.ws.rs.client is missing (what is correct)
2. plugin based product does not fail to resolve (whats wrong)

both test cases have a a requirement to the org.osgi.service.jaxrs bundle that requires package javax.ws.rs.client.

[laeubi]

@fipro78 unless the bug in Tycho is fixed, you must watch out for messages like this:

[INFO] Resolving dependencies of MavenProject: ....
[WARNING] Problems resolving provisioning plan.:
   Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; javax.ws.rs.client 0.0.0.
   Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; javax.ws.rs.core 0.0.0.
   Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; javax.ws.rs.sse 0.0.0.

these will give you the hints what is missing and needs to be added to the target for your project to package the product. If sou still see issues with none of these warnings please let us know so I can reopen this PR as further work will be performed in https://github.com/eclipse-tycho/tycho/pull/3393.