Elastic and Confluent partner to deliver an enhanced Kafka + Elasticsearch experience
Today, we are pleased to announce a partnership with Confluent to jointly develop and deliver an enhanced product experience to the Kafka-Elasticsearch community. Apache Kafka® is — and has been since the very early days — an important component of the Elastic ecosystem. The new partnership will work on co-developing new capabilities and improving existing product integrations to make it easier for our joint users to use the Elastic Stack and Kafka together, including strengthening the native integration between Elastic Cloud and Confluent Cloud.
Kafka and Elasticsearch, an integration that goes way back
Kafka and Elasticsearch have become integral components of modern enterprise data architectures. Right in the early days of Elastic, we saw huge demand from our community to natively support Kafka as an ingestion source for Elasticsearch. We saw evidence of this both through direct interactions with our users at events and in online forums. In response to the requests from the community, we developed native support for Kafka in Beats and Logstash, making Kafka a first-class citizen in our ingest pipeline. Kafka quickly became a common technology in Elastic architectures, with customers like CSX and Kroger using it as the primary tool for sending data to Elastic.
Kafka was also one of the first modules we developed in Metricbeat and Filebeat — since our centralized logging and monitoring customers also wanted to monitor the health and performance of their Kafka pipelines, along with the rest of the tech stack. Kafka continues to be one of our most popular integrations.
In partnership with Elastic, we have made it easier than ever to connect always-on, always-moving data flowing through Confluent to Elastic’s real-time search and analytics database. With our seamless integrations, organizations can set their data in motion for a new generation of use cases that were not possible before. - Jay Kreps, co-founder and CEO, Confluent
An exciting new chapter begins, building on a strong foundation
We are excited to build on this solid foundation and work directly with Confluent to further enhance the product experience for the joint user base. Elastic and Confluent both have deep ties to the community, and a shared mindset that encourages direct and open communication between our engineers and end users. We look forward to joining forces and investing R&D resources on both sides to bring you the best integrated product experience on various fronts, including enhancing the Confluent Cloud Elasticsearch Sink Connector that provides a direct integration between Elastic Cloud and Confluent Cloud, developing packaged joint solutions for specific use cases, and introducing easier ways to output data from Kafka in the Elastic Common Schema.
To learn more about the partnership, read the Confluent announcement from Jay Kreps, co-founder and CEO, Confluent.