IntelliJ IDEA support

[lppedd]

Hi all! I'm currently trying to understand why Tycho isn't working in IntelliJ IDEA, testing with https://www.eclipse.org/mat/. (see also issue https://youtrack.jetbrains.com/issue/IDEA-186628)

I have managed to debug the IDEA Maven implementation, and this is the call stack just before a dependency is not found.

At this point DependencyHelper#hasASolution returns false.

Tycho gets called as a Maven extension, via AbstractMavenLifecycleParticipant#afterProjectsRead

Question is, how should it be called? Which projects should be passed in the session? One at a time? All? I'm a bit ignorant on how Tycho works so I'm sorry if my question seems a bit stupid.

[laeubi]

Question is, how should it be called? Which projects should be passed in the session? One at a time? All?

It heavily depends on you project setup, but given you have a multi-module project with a common parent/aggregator it should be called on these projects.

If you call it on a single one it must work to call mvn at exactly this folder stand alone. If not you can again use mvn -pl -am but you need the tycho-build extension then!

[lppedd]

Hi @laeubi, thanks for the quick answer. So, I think we need to find an easier Tycho project to test with IDEA, MAT seems too complicated. Would https://github.com/eclipse/tycho/tree/master/demo/itp01 be ok?

But anyway

should be called on these projects

Are you referring to all the non-parent projects?

tycho-build extension

I think this should not be required so I'll leave this solution aside for now.

[laeubi]

Beside the demos, there are also a lot of integration test that could be used as an executable example.

I think this should not be required so I'll leave this solution aside for now.

Then IntelliJ must be "smart enough" to compute the set of required projects itself :-)

Are you referring to all the non-parent projects?

In a very simplified fashion Tycho works this way:

1. read all reactor projects
2. resolve the target platform(s)
3. resolve any dependencies between reactor projects and target platform content (probably taking pom specified stuff into account)
4. Injecting the dependencies into the maven model
5. Then the "normal" maven build takes over

So if you just run it on one project A, there might be the problem that you need also project B+C but Tycho (or more specifically maven) won't know about them and the build fails.

So either this one project must be self-contained (most probably not the case for any non trivial example), you need to know the required projects in advance (hard to guess for any non trivial example), or you use the tycho-build extension with -pl -am to compute the closure for your single project, see for this https://github.com/eclipse/tycho/tree/master/tycho-its/projects/reactor.makeBehaviour as an example and its corresponding test suite https://github.com/eclipse/tycho/blob/master/tycho-its/src/test/java/org/eclipse/tycho/test/reactor/makeBehaviour/MavenReactorMakeOptionsTest.java

As an alternative one can always build "all" projects (where all of course could be a subset of all possible maven projects in a given code repository).

Just another note: Its actually that ItelliJ do not support PDE than it not support Tycho as Tychos primary intend is to build PDE Artifacts with Maven so Eclipise is actually not using Tycho to build such a project!

[lppedd]

Ok I'll try to wrap my head around what you wrote, thanks 😄 Regarding PDE, IntellIJ supports PDE using the eclipse-pde-partial plugin, but Tycho support is needed to support automatic dependency resolution.

[laeubi]

Regarding PDE, IntellIJ supports PDE using the eclipse-pde-partial plugin, but Tycho support is needed to support automatic dependency resolution.

Automatic dependency resolution is what PDE does, so no Tycho there either (this is performed by P2).

Just let me know when anything is unclear.

[lppedd]

@laeubi so what you're saying is that I could run Maven via command line and then use the standard PDE support in IntelliJ?

[laeubi]

I don't know much about IntelliJ PDE support to guess if that works, but what I wanted to say is that any Plugin that claims to support PDE, should also support automatic dependency resolution based on a target platform and the manifest data. Otherwise it does not really supports PDE, but probably can read/write some of its meta-data :-)

[lppedd]

@laeubi oh ok! I mean, the plugin I linked supports reading the target configuration file and the manifest, and is capable of generating the correct IntelliJ module structure (I have used it for a big RCP application).

Being that Tycho is basically an extension to Maven, IntelliJ tries to setup the project using Maven, but fails.

[laeubi]

Alright, maybe then one should instruct InteliJ to not try using it as a maven project? I don't know how InteliJ setup the maven world, at laest you can do some testing yourself by issuing mvn dependency:tree in some folders to see whats going on at the maven/tycho level.

[laeubi]

By the way, currently there is a special property passed by m2e to a maven build that is read by Tycho to skip the extension in that case, is there also such property for intelij?

https://github.com/eclipse/tycho/blob/7b0ea29597e024fc71d850e82daa65d1e0b15801/tycho-core/src/main/java/org/eclipse/tycho/core/maven/TychoMavenLifecycleParticipant.java#L269-L275

Please don't use tycho.mode=maven as it is legacy and sceduled for removal:

• #676

[lppedd]

I'm doing some progress with debugging the demo project. That project points to https://download.eclipse.org/releases/2022-06/

<repositories>
 <repository>
   <id>eclipse</id>
   <layout>p2</layout>
   <url>https://download.eclipse.org/releases/2022-06</url>
 </repository>
</repositories>

And the P2 resolver uses some suffixes to look into the repository

However it cannot resolve it. Any idea what I'm missing?

[laeubi]

However it cannot resolve it. Any idea what I'm missing?

What is the exact error? Can you run the project on the command-line with mvn clean install ?

[lppedd]

What is the exact error?

mvn clean install

Yes it works 👀

[INFO] Fetching p2.index from https://download.eclipse.org/releases/2022-06/ (122B)
[INFO] Adding repository https://download.eclipse.org/releases/2022-06
[INFO] Fetching compositeContent.jar from https://download.eclipse.org/releases/2022-06/ (481B)
[INFO] Fetching compositeContent.xml from https://download.eclipse.org/technology/epp/packages/2022-06/ (442B)
[INFO] Fetching content.jar from https://download.eclipse.org/technology/epp/packages/2022-06/202206091200/ (87.05kB)
[INFO] Fetching p2.index from https://download.eclipse.org/releases/2022-06/202206151000/ (140B)
[INFO] Fetching content.xml.xz from https://download.eclipse.org/releases/2022-06/202206151000/ (792.86kB)
[INFO] Resolving dependencies of MavenProject: tycho.demo.itp01:tycho.demo.itp01:1.0.0-SNAPSHOT @ C:\Users\edoardo.luppi\Downloads\tycho-master\demo\itp01\tycho.demo.itp01\pom.xml

[laeubi]

Have you tried letting the code go on? Repositories are sadly queried on "best guess" sometimes and some failures are allowed, do you see the info print outs also in IntelliJ?

[lppedd]

It seems like there is no MetadataRepositoryFactory that actually checks remote URIs. All the ones I've seen check local JARs.

java.lang.NoClassDefFoundError: Could not initialize class org.apache.hc.client5.http.psl.PublicSuffixMatcherLoader

[laeubi]

I don't know how the InteliJ maven server works, but this seems something is not loaded/isolated as it should. Tycho start an internal OSGi Framework currently so if intelliJ reuses classloaders and/or mess up classpath you are maybe out of luck then.... but you can use mvnDebug to debug the commandline call and see what is different there compared to InteliJ.

[lppedd]

Got to the original error:

loader constraint violation: when resolving method 'org.slf4j.ILoggerFactory org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()' the class loader org.eclipse.osgi.internal.loader.EquinoxClassLoader @58d82974 of the current class, org/slf4j/LoggerFactory, and the class loader 'app' for the method's defining class, org/slf4j/impl/StaticLoggerBinder, have different Class objects for the type org/slf4j/ILoggerFactory used in the signature (org.slf4j.LoggerFactory is in unnamed module of loader org.eclipse.osgi.internal.loader.EquinoxClassLoader @58d82974, parent loader org.codehaus.plexus.classworlds.realm.ClassRealm @4f554509; org.slf4j.impl.StaticLoggerBinder is in unnamed module of loader 'app')

It seems there are two conflicting versions of SLF4J. Sad 😢

[laeubi]

This might indicate the IntelliJ do not properly isolate class realms for extensions here.

[lppedd]

@laeubi yeah, probably. Do you think workarounds exists for this issue? Like for example manually loading the same class files in the EquinoxClassLoader classloader.

[laeubi]

Not really, maybe IntelliJ is missing the core-exports are missing here but that's hard to guess without deeper knowledge of that code. But I think you now have a good test-case for the IntelliJ people to investigate further on this issue.

[bric3]

This might indicate the IntelliJ do not properly isolate class realms for extensions here.

To be fair, IJ isn't doing anything improperly, it's not OSGi based. So different trade-offs. As far as I know they IJ is based on pico container in regard of plugins.

[laeubi]

@bric3 that has nothing to do with OSGi or PicoContainer, the Tycho extension is starting an embedded OSGi framework (Maven itself do not use OSGi in any way...) and exports some stuff from its class realm into this framework, so if the classrealm is not setup property such issues can arise.

[lppedd]

@bric3 so there is a little difference here regarding how IntelliJ spawns the Maven server. It's not really a plugin, but it seems a new instance of the JVM is created.

https://github.com/JetBrains/intellij-community/blob/43bae47c6d4b65ded6969111034c172324dde329/plugins/maven/src/main/java/org/jetbrains/idea/maven/server/MavenServerManager.java#L474-L484

Also I see some code related to realms. https://github.com/JetBrains/intellij-community/blob/1e1f83264bbb4cb7ba3ed08fe0915aa990231611/plugins/maven/maven3-server-impl/src/org/jetbrains/idea/maven/server/Maven3XServerEmbedder.java#L164

[lppedd]

Then, a DefaultPlexusContainer spawned with that AppClassLoader is used to read lifecycle partecipants.

[lppedd]

Yep, confirmed.

[bric3]

@bric3 that has nothing to do with OSGi or PicoContainer, the Tycho extension is starting an embedded OSGi framework (Maven itself do not use OSGi in any way...) and exports some stuff from its class realm into this framework, so if the classrealm is not setup property such issues can arise.

Ah ok i understand your point, thanks for the clarification.

[lppedd]

I'm trying out again IDEA.
I'm receiving exceptions like this one:

java.lang.RuntimeException: Failed to load p2 repository with ID 'dtp' from location http://my-url:8081/nexus/content/repositories/zeus/
	at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.loadMetadataRepository(TargetPlatformFactoryImpl.java:297)
	at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.gatherExternalInstallableUnits(TargetPlatformFactoryImpl.java:267)
	at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.createTargetPlatform(TargetPlatformFactoryImpl.java:169)
	at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.createTargetPlatform(TargetPlatformFactoryImpl.java:134)
	at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.createTargetPlatform(TargetPlatformFactoryImpl.java:1)
	at org.eclipse.tycho.p2.manager.ReactorRepositoryManagerImpl.computePreliminaryTargetPlatform(ReactorRepositoryManagerImpl.java:90)
	at org.eclipse.tycho.p2.resolver.P2DependencyResolver.computePreliminaryTargetPlatform(P2DependencyResolver.java:224)
	at org.eclipse.tycho.core.resolver.DefaultTychoResolver.resolveProject(DefaultTychoResolver.java:125)
	at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.lambda$resolveProjects$0(TychoMavenLifecycleParticipant.java:157)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.WhileOps$1$1.accept(WhileOps.java:99)
	at java.base/java.util.Collections$2.tryAdvance(Collections.java:4747)
	at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:127)
	at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:502)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:488)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
	at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.resolveProjects(TychoMavenLifecycleParticipant.java:188)
	at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProjectsRead(TychoMavenLifecycleParticipant.java:114)
	at org.jetbrains.idea.maven.server.Maven3XServerEmbedder.loadExtensions(Maven3XServerEmbedder.java:1026)
	at org.jetbrains.idea.maven.server.Maven3XServerEmbedder.lambda$doResolveProject$1(Maven3XServerEmbedder.java:844)
	at org.jetbrains.idea.maven.server.Maven3ServerEmbedder$1.run(Maven3ServerEmbedder.java:367)
	at org.jetbrains.idea.maven.server.Maven3ServerEmbedder.executeWithSessionScope(Maven3ServerEmbedder.java:442)
	at org.jetbrains.idea.maven.server.Maven3ServerEmbedder.executeWithMavenSession(Maven3ServerEmbedder.java:375)
	at org.jetbrains.idea.maven.server.Maven3ServerEmbedder.executeWithMavenSession(Maven3ServerEmbedder.java:364)
	at org.jetbrains.idea.maven.server.Maven3XServerEmbedder.doResolveProject(Maven3XServerEmbedder.java:802)
	at org.jetbrains.idea.maven.server.Maven3XServerEmbedder.resolveProject(Maven3XServerEmbedder.java:767)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:359)
	at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200)
	at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:562)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:796)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:677)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:676)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.eclipse.equinox.p2.core.ProvisionException: Error while reading from repository: http://my-url:8081/nexus/content/repositories/zeus/site.xml.
	at org.eclipse.equinox.internal.p2.updatesite.UpdateSite.loadActualSiteFile(UpdateSite.java:255)
	at org.eclipse.equinox.internal.p2.updatesite.UpdateSite.load(UpdateSite.java:155)
	at org.eclipse.equinox.internal.p2.updatesite.metadata.UpdateSiteMetadataRepositoryFactory.initializeRepository(UpdateSiteMetadataRepositoryFactory.java:108)
	at org.eclipse.equinox.internal.p2.updatesite.metadata.UpdateSiteMetadataRepositoryFactory.load(UpdateSiteMetadataRepositoryFactory.java:62)
	at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.factoryLoad(MetadataRepositoryManager.java:63)
	at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:787)
	at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:685)
	at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:110)
	at org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadRepository(RemoteMetadataRepositoryManager.java:63)
	at org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadRepository(RemoteMetadataRepositoryManager.java:55)
	at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.loadMetadataRepository(TargetPlatformFactoryImpl.java:293)

The repository has been defined under repositories

<repository>
	<id>dtp</id>
	<url>http://my-url:8081/nexus/content/repositories/zeus/</url>
	<layout>p2</layout>
</repository>

Why is Equinox looking for a site.xml? Obviously I don't have a site.xml in that Nexus repository.

[laeubi]

Nexus has a known bug with producing wrong P2 Metadata, so you probably ant to check with an official repository first.

[lppedd]

@laeubi in the end it was still related to

java.lang.NoClassDefFoundError: Could not initialize class org.apache.hc.client5.http.psl.PublicSuffixMatcherLoader

What I'm doing now is use offline p2 repositories only, so no HTTP client is used.

[laeubi]

Then something seems wrong with how intelij setup the classpath for the plugin.