Doubling Down on Open

I am excited to announce that we will be opening the code of our X-Pack features - security, monitoring, alerting, graph, reporting, dedicated APM UIs, Canvas, Elasticsearch SQL, Search Profiler, Grok Debugger, Elastic Maps Service zoom levels, and machine learning - to facilitate greater collaboration with our customers and community in much the same way we do for our open source code today.

I am proud of the way our company has been built around our open source products, and hasn't compromised innovation or our commitment to openness along the way. As we look to the future, we saw an opportunity to double down on our belief in openness in even more fundamental ways, while introducing a new, more efficient model for building a successful, sustainable business around Open Source.

This blog outlines both the thinking and the details of the changes we are making, but let me make one thing clear - we are an open source company. We will remain an open source company. We are more open than we have ever been before and I, personally, and the team, in its entirety, are committed to this.

Watch the announcement from the Elastic{ON} 2018 keynote on 27 February 2018.

Why Open Source?

When I first began writing what became Elasticsearch, I knew that it had to be open source. Open source, as a development methodology and distribution model, provides the opportunity to reach so many more people. All of these people can contribute. Contribute through code, of course, but also by using freely available software and continually pushing the bounds of what’s possible.

Engaging with a community ensures that, when your project is successful, there is a group of passionate, dedicated developers who guide your feature development and drive the product forward into new and interesting areas. Take, for example, the introduction of aggregations into Elasticsearch that enabled usage as a scalable product for analytics. Projects like Kibana and Logstash, and later Beats were such inspiring uses of our technology that we joined forces with them. And as the needs of our users have become deeper and more specialized, we have found new ways to support them, with features and new products like machine learning, APM, and site search.

Our commitment to Open Source goes deep. It's where we spend the majority of our engineering effort, and our community of contributors and users are critical to our ongoing innovation. But driving these products forward as quickly as we do requires significant investment, and that's why we formed a company around this technology.

Why Commercial Software?

Why, then, would we build commercial software if we believe so deeply in the idea of open source software?

We are a business. And part of being a business is the belief that those businesses who can pay us, should. And those who cannot, should not be paying us. In return, our responsibility is to ensure that we continue to add features valuable to all our users and ensure a commercial relationship with us is beneficial to our customers. This is the balance required to be a healthy company.

One common open source business model, is to sell support subscriptions. Sadly, support-only business models trend towards a conflict between what's best for the user and what's best for the company. In these situations, the company isn’t motivated to make their products easier to use, more reliable, scalable because that would eat into their support profits. We were never, and never will be, willing to bear the tension of not making our software better in order to ensure that the company stays in business. We want to continuously improve and our goal with support is to make your project successful so that you are the expert in your Elastic Stack deployment. We want you to be successful so that you choose the Elastic Stack for your next projects too.

Another approach -- and if you've heard me speak you've probably heard me discuss this -- is to create an 'Enterprise Edition' of the software. This, in essence, results in a fracturing of the community and creates a divide between paying customers and ‘users’. It results in creating a version -- be it the Enterprise or Community Edition -- that that is deemed ‘authoritative’ and often lags master. A version that is missing features. A version tested and released on a different cycle. A version that, effectively, closes the open source code, as you are unable to know what was changed in order to enable the commercial features. At Elastic, all of our customers are open source users and using the same versions of our software products. We will not create a community vs. enterprise edition.

So what is left? To identify high-value features and to offer them as commercial extensions to the core software. This model, sometimes called 'open core', is what culminated in our creation of X-Pack. To build and integrate features and capabilities that we maintain the Intellectual Property (IP) of and offer either on a subscription or a free basis. Maintaining this control of our IP has been what has allowed us to invest the vast majority of our engineering time and resources in continuing to improve our core, open source offerings.

This has been the approach that has allowed us to build a distributed company of employees around the world who delight in being surprised by how the Elastic Stack is being used to solve real, tangible problems. To add features and capabilities that apply to our users and to develop some that allow us to sustain into the future.

But there are challenges with this approach…

More Open

Did you know that there is a free tier of X-Pack? Or that capabilities like Monitoring, Search Profiler, Grok Debugger and extra zoom levels in Elastic Maps are free for all users? Did you know that we continue to add more to this tier, including things like Canvas and Elasticsearch SQL?

When we introduced free features into X-Pack, we did it because we knew that these features would help make users more successful with the Elastic Stack. Unfortunately, too few of you were aware of -- or able to -- take advantage of these features. And it meant that a wide variety of users weren’t having the best experience with our software.

We also know that it is a problem that the process of obtaining this free software was a worse user experience involving a full cluster restart. And, if you wanted to see the code, you can’t. As these free features are relevant to a large amount of our users, you asked us good, and difficult, questions like: 'How do I interact with you on the free features?', 'How do I open an issue or contribute?'

And we didn't have a good answer.

By opening the code of X-Pack, we solve the issue of having some of our products open and others closed. Soon, for both free and commercial features you can open an issue, see the conversation, inspect the source code, collaborate with us, and open a pull request.

The Details

What does this mean, technically?

Updated on 2018-04-24 for clarity and to include a link to the Elastic License

As of the 6.3 release, in the $PRODUCT (Elasticsearch, Kibana, Beats, and Logstash) repositories:

• All existing Apache 2.0-licensed code maintains the same license, nothing changes.
• We will create a new X-Pack folder, and move the x-pack-$PRODUCT code into that folder under the Elastic License allowing for some derivative works and contribution.
• We will change the top-level license to simply note that some files in the repo are Apache 2.0 and some are covered by the Elastic License.

Also, X-Pack features will now be bundled into the default distribution. All free features are included and enabled by default and will never ‘expire’, and commercial features are opt-in via a trial license. The license for free features never expires,you no longer need to register to use these capabilities. In addition to this, an Apache 2.0-only distribution will be created for download.

More information, and an FAQ, can be found on the Opening X-Pack page.

In Summary

We believe in open source. As a distribution model. As a way to build a business. And as our future as a company. We commit and maintain that we will remain open and are excited that we will be MORE open in 6.3 than we have ever been before.

Thank you for your trust in us.