Elasticsearch: Java 9 and Beyond!

By
Nik Everett

Version 6.2 of the Elastic Stack will officially support running on Java 9 or Java 8. Your choice. Supporting Java 9 has been a pretty wild ride! Back in March of 2015 when we first got the Elasticsearch tests passing against Java 9 we didn't know what we were starting. After almost three years Java 9 has become an integral part of how we build Elasticsearch.

Following Development

We'vebeenkeepingElasticsearchcompatiblewithJava9foralongtimenow. You've been able to build Elasticsearch against Java 9, on and off, since March of 2015. We added Java 9 nodes to our test rotation in May of 2016 and with Java 9 build 118 we periodically upgraded those nodes to new preview releases as they were released. We wanted to keep current so that we understood the changes going into Java 9 and in case any of those changes to Java 9 had any unintended consequences for us. Just in case.

On upgrading to JDK 9 build 140 our testing paid off: we encountered a fairly significant performance regression. It hit our tests particularly hard with one jumping from 7 seconds to 31 seconds. Elasticsearch is fairly unique because it runs inside the Java Security Manager and its tests can generate a lot of files as it creates several in-process Elasticsearch nodes and acts on them to test the clustering mechanisms.

In what feels like a poster child for "open source development makes better software", the author of the JDK change that caused the performance regression joined our discussion of the issue and used our tests as a basis to propose a fix for Java 9 late in Java 9's release process.

Using Java 9

We've been excited to use the new Java 9 features and Painless has been taking advantage of Java 9's indified string concatenation since May of 2016 by generating bytecode with the Java 9 specific optimizations when running in Java 9 to speed up code like ctx._source['field'] + ' ' + ctx._source['other'].

Painless is somewhat special because it generates bytecode that it can customize based on the Java version. The rest of Elasticsearch didn't get to use Java 9 specific features until we multi-release JARs.

To build a multi-release JAR we compile most of the code for Java 8 and compile a second API compatible version of some classes for Java 9 and bundle them into a single JAR with the Java 9 classes tucked into a special directory that Java 9 reads and Java 8 ignores. This is the standard for multi-release JARs and seems like a pretty clever hack to me — not "UTF-8 is backwards compatible with 7 bit ASCII" clever, but still pretty clever.

Anyway, at first we proposed requiring developers to have both Java 8 installed to compile most of Elasticsearch and Java 9 installed to compile the Java 9 substituted classes. This felt like the most logical approach but we abandoned it because it forced all contributors to install both JDKs and set up two environment variables. That is just too much to expect from a first time contributor.

Luckily, Java 9 has wonderful support for compiling against Java 8. So we use it! Every build of Elasticsearch is built by Java 9's JDK. We still need to test against Java 8 and Java 9 but we don't require every contributor to do that. We added an optional system property, RUNTIME_JAVA_HOME, that you can set to a Java 8 JVM and the build will use that JVM for all of its testing. If you don't define RUNTIME_JAVA_HOME then the build will run the tests using the Java 9 JDK that compiled the code. Elastic will set this to a Java 8 JVM in some CI builds to make sure Elasticsearch has sufficient coverage, but contributors don't have to.

Plugin Developments

The build tools that Elasticsearch uses to build its plugins are published available as a Gradle plugin and they "come along for the ride" so to speak. The Java version checking and RUNTIME_JAVA_HOME finding, and static analysis changes pick up these changes in the 6.2.0 release and anyone using the Gradle plugin will also need Java 9 to build their Elasticsearch plugin. The philosophy of this Gradle plugin has always been to mirror the process used to build the plugins inside the Elasticsearch source code tree as closely as possible.

This may not be desired but we figure that anyone with super strong opinions about the build process used to build an Elasticsearch plugin likely is building their plugin using their own process rather than this plugin.

The Future

Java's change to six month release cycles has forced us to reconsider how we work with new Java versions. We very much want the Elastic Stack to be compatible with the new non-LTS releases as fast as possible because those releases will contain new features that can make Elasticsearch and the rest of the Elastic Stack faster. We also want to continue to support LTS releases so users are not constantly forced to upgrade the JVM that runs Elasticsearch.

Assuming Java continues its twice yearly JEP 322 release plan, we think that we will release a version of the Elastic Stack that is compatible with both the newest LTS release and the newest release. For example, we expect that the upcoming 6.2 will be compatible with Java 8 (the LTS release) and Java 9 (the newest release). Once Java 10 is released we'd like to release a version of Elasticsearch that is compatible with Java 8 and Java 10. Then a version that supports just Java 11 because it'll be both the newest LTS release and the newest release.

This compatibility plan is a long way from becoming a support policy. We can't guarantee that we're going to do what we planned. We followed Java 9's development fairly closely but were not compatible with it in 6.0 even though we very much wanted to be. That was the first Elastic Stack release after Java 9's release but we weren't able to get the compatibility until 6.2. We expect we'll be able to support six month releases of Java more quickly because, hopefully, they'll be smaller. And because we already support building a multi-release JAR. But we can't predict the future.

We are absolutely committed to making the best Elastic Stack possible but Java six month release cycle is going to have all kinds of surprises! We wanted to make our plans for compatibility public even if they aren't complete because we think users and contributors deserve to know what we're thinking. We really want to make the Elastic Stack compatible with the new versions as fast as possible so folks can use them and benefit from all the work put into the JVM. We also really want to keep the Elastic Stack compatible with the LTS releases of Java so folks can continue to use them without having to worry about keeping on the bleeding edge. We're not sure what this means regarding support yet.