druid icon indicating copy to clipboard operation
druid copied to clipboard

Java 17 compatibility

Open gianm opened this issue 3 years ago • 3 comments

Java 17 is the current LTS release. We'd like to officially support this.

gianm avatar Jul 30 '22 15:07 gianm

Some things we'll need to tackle:

  1. Strong encapsulation in Java 17 means that Druid requires various --add-opens and --add-exports parameters in order to function. Skipping these parameters leads to errors on startup, or, worse, an unstable experience. We need to provide these to the Java runtime without users having to explicitly configure them.
  2. DataSketches Memory does not currently support Java 14+ for all functionality. We use this both directly and via individual DataSketches. One specific incompatibility that I've noticed is memory-mapping files larger than 2GB in FrameFile. The unit tests explicitly skip this on Java 15: see FrameTestUtil#jdkCanDataSketchesMemoryMap. There may be other places as well.
  3. Run unit and integration tests on Java 17. (We can stop running in Java 15 when we do this; Java 15 isn't an LTS so there isn't much need to test both.)

gianm avatar Jul 30 '22 15:07 gianm

some related work that was already merged in support of this https://github.com/apache/druid/pull/12341 https://github.com/apache/druid/pull/12351 https://github.com/apache/druid/pull/12282 #12306

Upgrading Netty – while possibly not required – would be a nice to have, since Netty 3 does't officially support Java 17 AFAIK https://github.com/apache/druid/pull/12032

xvrl avatar Aug 02 '22 10:08 xvrl

Patch #12839 adds the relevant strong-encapsulation-related parameters to the command line when the Java version is >= 17, and updates the unit tests to run on Java 17 instead of 15.

gianm avatar Aug 10 '22 19:08 gianm

#12987 dodges the incompatibility between DataSketches Memory and JDK17. After speaking with the DataSketches team I learned it's safe to use the current version of DataSketches (and future 8+11 compatible versions) as long as we avoid allocations or memory maps using Memory.map, WritableMemory.map, and WritableMemory.allocateDirect.

gianm avatar Aug 27 '22 06:08 gianm

#12990 updated the docs to say that Java 17 is experimental.

As of now I am not aware of anything that is broken on Java 17. I use it on my dev machine and we have it running in various production clusters. I'd like to be able to start recommending it fully soon.

gianm avatar Dec 05 '22 08:12 gianm