Allow resolution of workspace dependencies without install

[lppedd]

As far as I understood, Tycho works by reading the MANIFEST/feature files, and it tries to match the dependencies found with the Target Platform, which in turn is calculated through different strategies (e.g. p2 repositories).

After some difficulties, I was able to adapt IntelliJ IDEA to work with Tycho (see also #1054); however, when refreshing the project model, local dependencies cannot be found. And by local I mean modules inside the same workspace.

An example of Tycho-provided dependencies:

---

From what I've read from the official docs and the internet, it seems Tycho tries to match the local dependencies with installed artifacts in the local .m2 repository. You understand that this becomes a bit of a limitation when working inside an IDE, as you cannot install each time you refresh the Maven tree.

A classic example is:

workspace
  > my.company.a
  > my.company.b

where my.company.b Require-Bundle my.company.a
The error that comes out if this is generally like the following:

[DEBUG] No solution found because the problem is unsatisfiable.: [Unable to satisfy dependency from my.company.b ; my.company.a 0.0.0.; No solution found because the problem is unsatisfiable.]
[INFO] {osgi.os=win32, osgi.ws=win32, org.eclipse.update.install.features=true, osgi.arch=x86}
[ERROR] Cannot resolve project dependencies:
[ERROR]   Software being installed: my.company.b
[ERROR]   Missing requirement: my.company.b requires 'my.company.a 0.0.0' but it could not be found
[ERROR] 
[ERROR] See https://wiki.eclipse.org/Tycho/Dependency_Resolution_Troubleshooting for help.

Now, what can we do to mitigate this? Could Tycho offer a switch for this kind of resolution? Is it somehow already available and my knowledge is too limited?

[laeubi]

And by local I mean modules inside the same workspace

There is no such thing as "workspace" in maven, but intellij should be able to provide a WorkspaceReader that is asked for workspace related artifacts but this requires using maven model.

So as explained intellij must either be able to compute the full project closure or support PDE directly.

[lppedd]

There is no such thing as "workspace" in maven

Yes, by workspace I meant the concept of multiple plugins/features developed in the same context.

but intellij should be able to provide a WorkspaceReader that is asked for workspace related artifacts

So what I've described is actually possible, all I'd need to do is try to understand how to correctly provide the overall workspace artifacts to Tycho?
Probably what you're saying is the same as what the original author of this issue was mentioning, right?

To make Tycho work properly, whole project tree should be passed to session.setAllProjects and session.setProjects here instead of currently resolving project

[mickaelistria]

a WorkspaceReader that is asked for workspace related artifacts but this requires using maven model.

I'm not sure that would be enough: the resoluti step here and the error come from p2; and the WorkspaceReader is not used during p2 resolution. Idea already provides a WorkspaceReader ( https://github.com/JetBrains/intellij-community/search?q=WorkspaceReader ) so I don't think a new one is necessary. However, the WorkspaceReader can participate to Maven resolution, and if there is no Maven instruction (eg a pom.xml dependency) on how to locate the my.company.a bundle, the WorkspaceReader cannot help. It's pretty similar to trying to build the my.company.b module only from CLI.

[laeubi]

I'm not sure that would be enough: the resoluti step here and the error come from p2; and the WorkspaceReader is not used during p2 resolution.

As mentioned it would require to declare dependencies in the maven model as Tycho otherwise can't know what requirement matches an artifact. "automatic" discovery currently only work if the project is build in the same reactor.

The WorkspaceReader cannot help

If one want it more "dynamic" I think we require the help of a BuildContext but currently there is no way to list workspace projects (what is something that Tycho actually will require) I opened https://github.com/eclipse-m2e/m2e-core/discussions/876 for a discussion about that aproach

It's pretty similar to trying to build the my.company.b module only from CLI.

Given this is all the same multi-module project, using the Tycho build extension one could use

mvn -am -pl :my.company.b

But to be fair, m2e actually suffer from the same problem and I have already did some investigations how to archive this but has not yet get enough need for this. If this is crucial to someones business and likes to speed up the development in that area a sponsoring would allow me to assign more time-slots particular issue, or contact me directly for a specialized offer to implement certain feature in Tycho or somewhere else.

[lppedd]

@mickaelistria there is a point where we can change what IntelliJ IDEA does tho. In the screenshot you can see that the IntelliJ Maven integration is passing only a single project to the session, so that might contribute to this behavior indeed.

The other info that you both wrote afterwards are a bit out of my knowledge for now, but I think I more or less understood.

[laeubi]

IntelliJ Maven integration is passing only a single project to the session

As mentioned above, technically IntelliJ should/could call the GraphBuilder with a project list of "project" and the MAKE_DOWNSTREAM behavior, but this would require it to support using core extensions as well.

[lppedd]

Why is MAKE_DOWNSTREAM important? I can't seem to find info on make behaviors.
I can probably pass in the correct list of projects.

[laeubi]

Why is MAKE_DOWNSTREAM important?

Because you want all projects your current project depends on.

I can probably pass in the correct list of projects.

If you can compute the "correct list" then you don't need this...

[lppedd]

MAKE_DOWNSTREAM does not seem to work.

You did mention:

"automatic" discovery currently only work if the project is build in the same reactor.

And I suppose you're referring to this:

https://github.com/eclipse-tycho/tycho/blob/98289ada2ce0a177b9c4c5d4872b4658631d7a0b/tycho-core/src/main/java/org/eclipse/tycho/core/maven/TychoMavenLifecycleParticipant.java#L106-L114

So, projects are passed in via the MavenSession. And that means we're back at the IntelliJ session.setProject(...) call (screenshot above). Do you think I can try this way and pass in the projects manually?

[laeubi]

MAKE_DOWNSTREAM does not seem to work.

The it seems you are doing something wrong ;-)

Do you think I can try this way and pass in the projects manually?

There is nothing that forbids you to do so...

[lppedd]

The it seems you are doing something wrong ;-)

Yeah most probably 😄 It's pretty intricated and me not being an expert in any of them (well, maybe a bit in IDEA) means I have to debug to find how everything work together. Might have missed a step.

There is nothing that forbids you to do so...

In this case do I still have to keep MAKE_DOWNSTREAM or can I remove the setMakeBehavior call?

[laeubi]

Make behavior is only relevant for the GraphBuilder and you need to enable the Tycho build extension for it to work, the plain **Tycho-Maven-Project-**Extension do not need nor can do anything with that..

[lppedd]

Thanks for the tip. To enable the extension do I have to edit the .mvn/extensions.xml file in the workspace root?

Also, I have tried passing in all the projects at the same time (I have three parent multimodule projects, 45 projects in total) and dependency resolution works! The only problem is that it takes 10 minutes every time to re-import. Need to investigate why.

Edit: avg. 34 minutes with 75 projects

[lppedd]

I have tried using the tycho-build extension, by placing the Jar inside %MAVEN_HOME%/lib/ext. However, I receive this exception when trying mvn initialize -X

[WARNING] Error injecting: org.eclipse.tycho.build.TychoGraphBuilder
java.lang.NoClassDefFoundError: org/eclipse/tycho/core/shared/MavenLogger
    at java.lang.Class.getDeclaredConstructors0 (Native Method)
    at java.lang.Class.privateGetDeclaredConstructors (Class.java:3137)
    at java.lang.Class.getDeclaredConstructors (Class.java:2357)
    at com.google.inject.spi.InjectionPoint.forConstructorOf (InjectionPoint.java:245)
    at com.google.inject.internal.ConstructorBindingImpl.create (ConstructorBindingImpl.java:115)
    at com.google.inject.internal.InjectorImpl.createUninitializedBinding (InjectorImpl.java:706)
    at com.google.inject.internal.InjectorImpl.createJustInTimeBinding (InjectorImpl.java:930)
    at com.google.inject.internal.InjectorImpl.createJustInTimeBindingRecursive (InjectorImpl.java:852)
    at com.google.inject.internal.InjectorImpl.getJustInTimeBinding (InjectorImpl.java:291)
    at com.google.inject.internal.InjectorImpl.getBindingOrThrow (InjectorImpl.java:222)
    at com.google.inject.internal.InjectorImpl.getProviderOrThrow (InjectorImpl.java:1040)
    at com.google.inject.internal.InjectorImpl.getProvider (InjectorImpl.java:1071)
    at com.google.inject.internal.InjectorImpl.getProvider (InjectorImpl.java:1034)
    at com.google.inject.internal.InjectorImpl.getInstance (InjectorImpl.java:1086)
    at org.eclipse.sisu.space.AbstractDeferredClass.get (AbstractDeferredClass.java:48)
    at com.google.inject.internal.ProviderInternalFactory.provision (ProviderInternalFactory.java:85)
    at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision (InternalFactoryToInitializableAdapter.java:57)
    at com.google.inject.internal.ProviderInternalFactory$1.call (ProviderInternalFactory.java:66)
    at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:112)
    at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:127)
    at com.google.inject.internal.ProvisionListenerStackCallback.provision (ProvisionListenerStackCallback.java:66)
    at com.google.inject.internal.ProviderInternalFactory.circularGet (ProviderInternalFactory.java:61)
    at com.google.inject.internal.InternalFactoryToInitializableAdapter.get (InternalFactoryToInitializableAdapter.java:47)
    at com.google.inject.internal.ProviderToInternalFactoryAdapter.get (ProviderToInternalFactoryAdapter.java:40)
    at com.google.inject.internal.SingletonScope$1.get (SingletonScope.java:168)
    at com.google.inject.internal.InternalFactoryToProviderAdapter.get (InternalFactoryToProviderAdapter.java:39)
    at com.google.inject.internal.InjectorImpl$1.get (InjectorImpl.java:1050)
    at org.eclipse.sisu.inject.LazyBeanEntry.getValue (LazyBeanEntry.java:81)
    at org.eclipse.sisu.plexus.LazyPlexusBean.getValue (LazyPlexusBean.java:51)
    at org.eclipse.sisu.plexus.PlexusRequirements$RequirementProvider.get (PlexusRequirements.java:250)
    at org.eclipse.sisu.plexus.ProvidedPropertyBinding.injectProperty (ProvidedPropertyBinding.java:48)
    at org.eclipse.sisu.bean.BeanInjector.injectMembers (BeanInjector.java:52)
    at com.google.inject.internal.MembersInjectorImpl.injectMembers (MembersInjectorImpl.java:160)
    at com.google.inject.internal.ConstructorInjector.provision (ConstructorInjector.java:124)
    at com.google.inject.internal.ConstructorInjector.access$000 (ConstructorInjector.java:32)
    at com.google.inject.internal.ConstructorInjector$1.call (ConstructorInjector.java:98)
    at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:112)
    at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:127)
    at com.google.inject.internal.ProvisionListenerStackCallback.provision (ProvisionListenerStackCallback.java:66)
    at com.google.inject.internal.ConstructorInjector.construct (ConstructorInjector.java:93)
    at com.google.inject.internal.ConstructorBindingImpl$Factory.get (ConstructorBindingImpl.java:306)
    at com.google.inject.internal.InjectorImpl$1.get (InjectorImpl.java:1050)
    at com.google.inject.internal.InjectorImpl.getInstance (InjectorImpl.java:1086)
    at org.eclipse.sisu.space.AbstractDeferredClass.get (AbstractDeferredClass.java:48)
    at com.google.inject.internal.ProviderInternalFactory.provision (ProviderInternalFactory.java:85)
    at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision (InternalFactoryToInitializableAdapter.java:57)
    at com.google.inject.internal.ProviderInternalFactory$1.call (ProviderInternalFactory.java:66)
    at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:112)
    at org.eclipse.sisu.bean.BeanScheduler$CycleActivator.onProvision (BeanScheduler.java:230)
    at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:120)
    at com.google.inject.internal.ProvisionListenerStackCallback.provision (ProvisionListenerStackCallback.java:66)
    at com.google.inject.internal.ProviderInternalFactory.circularGet (ProviderInternalFactory.java:61)
    at com.google.inject.internal.InternalFactoryToInitializableAdapter.get (InternalFactoryToInitializableAdapter.java:47)
    at com.google.inject.internal.ProviderToInternalFactoryAdapter.get (ProviderToInternalFactoryAdapter.java:40)
    at com.google.inject.internal.SingletonScope$1.get (SingletonScope.java:168)
    at com.google.inject.internal.InternalFactoryToProviderAdapter.get (InternalFactoryToProviderAdapter.java:39)
    at com.google.inject.internal.InjectorImpl$1.get (InjectorImpl.java:1050)
    at org.eclipse.sisu.inject.LazyBeanEntry.getValue (LazyBeanEntry.java:81)
    at org.eclipse.sisu.plexus.LazyPlexusBean.getValue (LazyPlexusBean.java:51)
    at org.codehaus.plexus.DefaultPlexusContainer.lookup (DefaultPlexusContainer.java:263)
    at org.codehaus.plexus.DefaultPlexusContainer.lookup (DefaultPlexusContainer.java:255)
    at org.codehaus.plexus.DefaultPlexusContainer.lookup (DefaultPlexusContainer.java:249)
    at org.apache.maven.cli.MavenCli.container (MavenCli.java:676)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:282)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: java.lang.ClassNotFoundException: org.eclipse.tycho.core.shared.MavenLogger
    ...

Am I missing another Jar?
Probably it's better adding it under .mvn, but that also mean I have to add it under multiple directories.

---

I've tried both under the .mvn folder, and under the build element.
None seems to work.

[laeubi]

I have tried using the tycho-build extension, by placing the Jar inside %MAVEN_HOME%/lib/ext

this is not supported, you need to use .mvn folder, sharing a minimal example would help to give further assistance.

[lppedd]

I'll try to create a small example with a couple plugins.
What I can already tell you is that it seems the IntelliJ resolution process never invokes the TychoGraphBuilder. It is created (I've debugged it), but never called.

[laeubi]

What I can already tell you is that it seems the IntelliJ resolution process never invokes the TychoGraphBuilder. It is created (I've debugged it), but never called.

As I previously said, InteliJ most likely needs to be enhanced to do so.

[lppedd]

As I previously said, InteliJ most likely needs to be enhanced to do so.

This is better done by a real IntelliJ developer, not by an amateur (me lol).
Let's say I'll go forward with passing in the required projects in the same reactor: as of now I'm always passing in all the projects, but obviously this takes a lot of time even for a small change (e.g. an added required bundle).
To quote you again

"automatic" discovery currently only work if the project is build in the same reactor

This means if my.company.c needs to be refreshed, and it depends solely on my.company.b, then I need to pass in only those two, right? Or do I also have to include the complete POM hierarchy of my.company.b, plus its dependencies, and so on?

[lppedd]

Support for IntelliJ IDEA is implemented. There is an issue with logging tho. You can see what the issue is about here. Any suggestion to get it working is much appreciated! 😄