ReactiveX
Maven poms have depencies as runtime instead of compile.
I've been scratching my head this evening after switching over to the rxnetty project and it looks like the poms since 0.4.0 all have dependencies as runtime rather than compile. I can't immediately see why this would be happening, since the gradle files point to compile.
https://repo1.maven.org/maven2/io/reactivex/rxnetty/0.4.4/rxnetty-0.4.4.pom
Are you depending on rxnetty and having a problem because the dependencies are in the runtime scope?
From a dependency management point of view, there is no difference between runtime and compile scopes, you always need both. The differentiation is only relevant when building a project, but for rxnetty we're using Gradle and not Maven, so the contents of the pom don't affect the build.
Yes. I am trying to build a simple project with maven where I should only need to depend on rxnetty and get the rest of what I need from dependencies. With the rxnetty pom on maven central set up the way it is, I need to explicitly include other dependencies.
"From a dependency management point of view, there is no difference between runtime and compile scopes, you always need both."
This is not necessarily the case. You do not need runtime dependencies at compile time (for instance an spi impl like log4j-slf4j) but you do need compile dependencies at runtime. This is why runtime dependencies aren't included in the compile classpath (in maven).
I'm fairly certain that if you took a gradle project and tried to depend only on rxnetty (io.reactivex 0.4.0) you wouldn't be able to compile rx.Observable or the netty codec stuff. There's no reason for the lifecycle to be that different.
This seems to be new behavior in the 0.4.0 project, so maybe something changed with the pom generation from gradle?
https://repo1.maven.org/maven2/com/netflix/rxnetty/rx-netty/0.3.18/rx-netty-0.3.18.pom
It's late here, so I can't debug any further. I'm assuming there are some shared gradle components used to generate poms, since a simple grep on the project doesn't turn up too much.
Here is a sample mvn pom:
<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>io.reactivex</groupId>
<artifactId>rxnetty-example</artifactId>
<version>0.4.4</version>
<dependencies>
<dependency>
<groupId>io.reactivex</groupId>
<artifactId>rxnetty</artifactId>
<version>0.4.0</version>
<scope>compile</scope>
</dependency>
</dependencies>
<name>rxnetty-example</name>
</project>
When I run 'mvn dependency:tree', I can see the transitive dependencies coming through just fine:
23:49 $ mvn dependency:tree
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building rxnetty 0.4.4
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ rxnetty-example ---
[INFO] io.reactivex:rxnetty-example:jar:0.4.4
[INFO] \- io.reactivex:rxnetty:jar:0.4.0:compile
[INFO] +- io.netty:netty-codec-http:jar:4.0.21.Final:runtime
[INFO] | +- io.netty:netty-codec:jar:4.0.21.Final:runtime
[INFO] | \- io.netty:netty-handler:jar:4.0.21.Final:runtime
[INFO] +- io.netty:netty-transport-native-epoll:jar:4.0.21.Final:runtime
[INFO] | +- io.netty:netty-common:jar:4.0.21.Final:runtime
[INFO] | +- io.netty:netty-buffer:jar:4.0.21.Final:runtime
[INFO] | \- io.netty:netty-transport:jar:4.0.21.Final:runtime
[INFO] +- io.reactivex:rxjava:jar:1.0.0-rc.10:runtime
[INFO] \- org.slf4j:slf4j-api:jar:1.7.6:runtime
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.393 s
[INFO] Finished at: 2014-12-07T23:49:09-08:00
[INFO] Final Memory: 11M/245M
[INFO] ------------------------------------------------------------------------
I tried it with gradle too:
repositories { jcenter() }
apply plugin: 'java'
dependencies {
compile 'io.reactivex:rxnetty:0.4.0'
}
With "gradlew dependencies --configuration compile", I can see the appropriate transitive dependencies:
23:53 $ gradlew dependencies --configuration compile
:dependencies
------------------------------------------------------------
Root project
------------------------------------------------------------
compile - Compile classpath for source set 'main'.
\--- io.reactivex:rxnetty:0.4.0
+--- io.netty:netty-codec-http:4.0.21.Final
| +--- io.netty:netty-codec:4.0.21.Final
| | \--- io.netty:netty-transport:4.0.21.Final
| | \--- io.netty:netty-buffer:4.0.21.Final
| | \--- io.netty:netty-common:4.0.21.Final
| \--- io.netty:netty-handler:4.0.21.Final
| +--- io.netty:netty-buffer:4.0.21.Final (*)
| +--- io.netty:netty-transport:4.0.21.Final (*)
| \--- io.netty:netty-codec:4.0.21.Final (*)
+--- io.netty:netty-transport-native-epoll:4.0.21.Final
| +--- io.netty:netty-common:4.0.21.Final
| +--- io.netty:netty-buffer:4.0.21.Final (*)
| \--- io.netty:netty-transport:4.0.21.Final (*)
+--- io.reactivex:rxjava:1.0.0-rc.10
\--- org.slf4j:slf4j-api:1.7.6
(*) - dependencies omitted (listed previously)
BUILD SUCCESSFUL
Total time: 1.323 secs
Do those examples capture what you're talking about? Are you seeing different behavior?
Caveat emptor, Compiling against rx.Observable references without it being a direct compile dependency is risky, you're essentially relying on a transitive dependency to bring in a module.
I did some research and a co-worker explained what is happening to you. When dependencies are in the compile scope in a POM, they are then added to your compile scope. We believe pretty strongly that if you need something on your compile classpath you should have a direct dependency on it. This becomes an issue of how a module should convey what their feel should be part of their public API and what is an implementation detail. It's too bad our module descriptors don't have a way to convey this, other than munging them runtime/compile scope.
I understand the desire to manage things more explicitly, but this goes against the nature of the system and you will be plunging everyone back into dependency management hell.
If all netflix projects start publishing to maven central with second-order dependencies as runtime, starting up a project will get very difficult.
Let's take ribbon for example. I can see that I might be using rxjava in my own code anyway, so I might have that. But the example linked to from the readme:
https://github.com/Netflix/ribbon/blob/master/ribbon-examples/src/main/java/com/netflix/ribbon/examples/rx/template/RxMovieTemplateExample.java
has classes from both rxnetty and (parts of) netty. Am I going to have to go through and figure out all of the nested compile dependencies that are now set to runtime (and not in the instructions for starting a ribbon project)?
My build (and my IDE) will be mysteriously broken because these POMs don't follow the conventions for everything else on maven central.
If you want to enforce this behavior for internal projects and repositories, that is one thing, but it is just going to break projects pulling from maven central.
I agree with dstengle that making the netty dependencies runtime puts unnecessary responsibility on the developer using RxNetty to know what version of netty was used. RxNetty exposes the netty types in its public API so they should be resolvable at compile time.
I started searching for this issue because I wanted to write the following code
RxClient<io.netty.buffer.ByteBuf, io.netty.buffer.ByteBuf> tcpClient = RxNetty.createTcpClient("someserver.somedomain.com", 9600);
but I can't without identifying the module containing ByteBuf and adding the same version of the module to my project dependency list. This works but it is brittle and I need to manually make sure I keep in sync with changes to the RxNetty project.
I looked at some of the projects that depend on rxjava like rxjava-string and their dependency on rxjava is compile time and in that specific case it depends on a fairly old version of rxjava.
What do you think?
Thanks
dan finucane
I agree with @dstengle that he should not be providing these dependencies explicitly as it is part of rxnetty API (ByteBuf usage i.e.)
@quidryan thoughts?
I understand the point. The problem is that we're conflating what is needed to compile rxnetty and what is part of its public interface. Ideally I'd want way to differentiate those two things. Either way, the Gradle publishing code isn't very flexible in this regard, and will a require changes to make which configuration to publish dependencies in configurable. I don't have any immediate plans to implement something like this, but I'm not opposed to it.
@quidryan
When you mean conflating compile and the public api, do you specifically mean something like slf4j, which is literally never in a signature and only needed in the runtime classpath?
So the distinction would be pom-for-self and pom-for-export in a sense (even though you aren't using maven here)?
I'll add that I've had issues compiling against Karyon's Jersey module for this same issue. It invites you to extend a class that has ByteBuf generics (though my own code does not), and then compilation fails for the lack of RxNetty at compile time. So you may not even be aware RxNetty is involved in that project until you get those seemingly bizarre compile errors.
+1
This runtime scope defeats the purpose of using something like maven.
I wanted to create some standalone server to test out RxNetty and RxJava so here's what I tried to do:
<dependencies>
<dependency>
<groupId>io.reactivex</groupId>
<artifactId>rxjava</artifactId>
<version>1.0.8</version>
</dependency>
<dependency>
<groupId>io.reactivex</groupId>
<artifactId>rxnetty</artifactId>
<version>0.4.8</version>
</dependency>
</dependencies>
But i end up needing to add some dependencies I almost shouldn't even care about:
<dependencies>
<dependency>
<groupId>io.reactivex</groupId>
<artifactId>rxjava</artifactId>
<version>1.0.8</version>
</dependency>
<dependency>
<groupId>io.reactivex</groupId>
<artifactId>rxnetty</artifactId>
<version>0.4.8</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-codec-http</artifactId>
<version>4.0.25.Final</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-transport-native-epoll</artifactId>
<version>4.0.25.Final</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.6</version>
</dependency>
</dependencies>
I added those two cause they are the top level under my dependencies and hopefully all their dependencies are using the compile scope
Adding a +1 for this issue to be addressed as well.
I had the same problem (and resolution) as @Crystark. Dependencies like (rx.Observable and io.netty.buffer.ByteBuf) that are part of the public API should be included in the compile scope so that the end user does not have to track down what internal dependencies RxNetty needs
I suppose one workaround might be to set up a separate build and publish infrastructure to automatically create a importable POM that has the kind of scope settings every other project has. Would that be a good use of effort?
just bumped into this, too.
This isn't idiomatic: the primary consumer of a pom is maven, it should be marked compile as we can't use it otherwise. If folks are looking for BOM support (eg to correlate versions via a maven import) that's a different concern than marking things runtime when you actually have a hard dep on it.
The impact of this is basically someone tries to make a simple example, then has to hunt for this issue to figure out why it won't work. This bleeds value from the quick start story for folks beyond gradle. Even if people love gradle, if they love developers they shouldn't make maven's pom.xml useless.
<!-- lines of dependency config needed for this line of code:
server = HttpServer.newServer().start((req, resp) -> resp.flushOnlyOnReadComplete());
-->
<dependency>
<groupId>io.reactivex</groupId>
<artifactId>rxnetty-http</artifactId>
<version>0.5.2-rc.4</version>
</dependency>
<dependency>
<groupId>io.reactivex</groupId>
<artifactId>rxnetty-common</artifactId>
<version>0.5.2-rc.4</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-buffer</artifactId>
<version>4.1.5.Final</version>
</dependency>
<dependency>
<groupId>io.reactivex</groupId>
<artifactId>rxjava</artifactId>
<version>1.1.5</version>
</dependency>
There is a nebula.compile-api plugin that will let you specify dependencies that should go into the compile scope.
Related ticket: https://github.com/nebula-plugins/nebula-publishing-plugin/issues/31
