This Week in Elasticsearch - April 30, 2014
Welcome to This Week in Elasticsearch. In this roundup, we try to inform you about the latest and greatest changes in Elasticsearch. We cover what happened in the GitHub repositories, as well as many Elasticsearch events happening worldwide, and give you a small peek into the future of the project.
Elasticsearch core
- Field data: Improved circuit breaker error messages to include name of the field that caused a circuit break (#5718, master and 1.x)
- Field data: Code cleanup, removed unused or almost unused methods (#5874, master and 1.x)
- Field data: Use segment ordinals as global ordinals when possible (#5873)
- Field data: Made ordinals start from
0
(#5871, master and 1.x) - Field data: Improved global ordinals on low cardinality fields (#5854, master and 1.x)
- Field data: Provided better error message if field has no field data type (#5979, master and 1.x)
- Lucene: Enabled turning on
IndexWriter
's InfoStream (#5891, master and 1.x) - Lucene: Upgraded to Lucene 4.8 (#5932, master and 1.x)
- Document versioning: Versioned get operations tests for version equality in all version types (#5929, master and 1.x)
- Document versioning: Throw a request validation error when calling the update API with the
external
&external_gte
version type as their usage can lead to data loss (#5929, master and 1.x) - Internal: Made Create/Update/Delete classes less mutable (commit, master and 1.x)
- Internal: Don't lookup version for create operations with auto-generated id (#5917)
- Internal: Upgraded to Guava 17 (#5953, master and 1.x)
- Internal: Upgraded to Jackson 2.3.3 (commit, master and 1.x)
- Search: Fail a
DFS_QUERY_THEN_FETCH
search request if all shards failed the query phase (commit, master and 1.x) - Query DSL: Thrown exception if an additional field was placed inside the query (#4913, master and 1.x)
- Query DSL: Added support for multiple field values in function score query (#5940, master and 1.x)
- Query DSL: Disabled parent/child queries in delete by query api as they can cause index corruptions (#5916, master and 1.x)
- Query DSL: Made use of global ordinals in parent-child queries (#5846, master and 1.x)
- Query DSL: Range/Term query/filter on dates fail to handle numbers properly (#5969, master, 1.x and 1.1)
- Query DSL: Fixed sorting on nested documents (#5986, master, 1.x, 1.1 and 1.0)
- Search templates:
size
andfields
parameters were ignored if placed after the query in a search template (#5933, master, 1.x and 1.1) - Scripting: Disabled dynamic scripting by default (#5853, master and 1.x)
- Aggregations: Removed abstraction that allowed to plug in different percentiles estimators (#5859, master and 1.x)
- Aggregations: Delayed creation of aggregations/facets to
QUERY
phase for better object recycling (#5821, master and 1.x) - Aggregations: Improved bucket creation if the bucket already exists (#5955, master and 1.x)
- Aggregations: Improved parsing of aggregations, made it more strict (#5827, master and 1.x)
- Aggregations: Added global ordinal based implementation for significant terms aggregation (#5970, master and 1.x)
- Aggregations: Improved the way sub-aggregations are collected (#5975, master and 1.x)
- Cat apis: Cat allocation returns now no value instead of
-1
fordisk.total
when not available (#5948, master and 1.x) - Cat apis: Cat thread_pool returns now the threadpool type and the configured sizes (#5366, master and 1.x)
- Testing: Increased upper bound for number of replicas used in our tests: up to the number of the data nodes - 1 instead of 1 (# #5896, master and 1.x)
- Testing: Added test that verifies no duplicates are created (commit, master and 1.x)
- Testing: Added support for compatibility mode to our test infrastructure so that specific test features (e.g. randomization of number of shards) can be disabled via system property (#5897, master and 1.x)
- Testing: Randomly introduced a client node in test clusters (#5949, master and 1.x)
- Testing: Used
URI
rather thanURL
to access files from classpath (#5950, master and 1.x)
Elasticsearch Ecosystem
Here's some more information about what is happening in the ecosystem we are maintaining around the ELK stack - that's Elasticsearch plus Logstash and Kibana - including plugin and driver releases.
- The latest version of the Couchbase plugin for Elasticsearch has been released.
- It's good to be loved! Apparently, AWS super users prefer Elasticsearch.
- Shay Banon wrote a post on resiliency, Elasticsearch, plus some insights into our testing processes. Forget Chaos Monkey, we've got evil tests!
- Check out how Elasticsearch powers The Guardian's newsroom, starring Graham Tackley, Director of Architecture for the Guardian, and Shay Banon, our CTO.
- Alexander Reelsen shared an in-depth how to on using the ELK stack to enrich your searches with geodata.
- Luca Cavanna posted all the news you can use on aliases and increased ease of use for them in 1.1.
- Alexander Reelsen treated us to a look at keeping search requests short and sweet.
- Florian Hopf posted an overview of the Significant Terms aggregation.
- Bastian Widmer shares some points on why visualizing your logs is a great idea, and why Kibana is the best tool to do so.
- Brian Altenhofel shares a how to on using Elasticsearch with Drupal 7.
Slides & Videos
Jordan Sissel's presentation on the ELK stack from the OSDC 2014 conference
Boaz Leskes' presentation from NoSQL Matters Cologne 2014
Luca Cavanna on new features in 1.1 and Boaz Leskes treats us to an Elasticsearch Marvel Demo
Where to find Us
We'd love to feature all the great Elasticsearch, Logstash and Kibana presentations and meetups happening worldwide in this section. If you're speaking or hosting a meetup, let our Community Manager, Leslie Hawthorn, know!
Austria
The Elasticsearch Vienna Meetup Group just formed! Join the group now to get updates on their first meeting.
Brasil
- Elasticsearch is very excited to send our first speaker from the company to Brasil! Leslie Hawthorn, our Community Manager, will be returning to FISL once again to speak on Community 2.0: Beyond Using Software Livre. Leslie will speak on May 7th and FISL 15 runs from May 7-10th in Porto Alegre. Her talk will be in English but most of the talks on the program are given in BR-PT. Leslie will also be on hand to answer questions you may have about Elasticsearch, Logstash and Kibana.
- During her visit for FISL 15, Leslie will also speak at the first ever Rio de Janiero Elasticsearch Meetup on May 5th.
- If you are attending FISL 15 or make your home in or near Porto Alegre, please join us for the first Porto Alegre Elasticsearch Meetup on May 8th..
Bulgaria
Honza Kral will discuss how to Explore Your Data using Elasticsearch at the Bulgarian Web Summit 2014. The conference takes place on May 31st in Sofia.
France
- Honza Kral will be speaking at DjangoConEU on From __icontains to search. The conference takes place May 13-17th on the Île des Embiez in France.
- The sixth Elasticsearch France Meetup will take place on May 14th at Nuxeo's offices in Paris. Doors open at 7:30 PM on May 14th.
- David Pilato will be attending dotSCale on May 19th in Paris. Don't miss his dotScale workshop, Elasticsearch Overview, on May 17th!
- David Pilato will host an Elasticsearch workshop at the Solutions Linux Conference on May 20th. The conference runs from May 20-21st in Paris. If you don't have time to attend David's workshop, make sure to stop by the Elasticsearch booth to say hello!
- David Pilato will run a workshop on Elasticsearch and Kibana at the Breizhcamp 2014. The event runs from May 21-23rd in Rennes, and David's workshop takes place on May 22nd.
Germany
- Alexander Reelsen will be speaking at the Search Meetup Karslruhe on May 6th. Doors open at 7:15 PM.
- Our friends at XING have created the Hamburg Elasticsearch User Group and will welcome Alexander Reelsen at the first meeting on Wednesday, May 7th. Alex will cover Using Elasticsearch, Logstash & Kibana to create realtime dashboards. Doors open at 6:30 PM.
- The Elasticsearch Stuttgart Meetup will host their second meeting on May 26th. Attendees can look forward to hearing about Elasticsearch on AWS or Google Compute Engine. Doors open at 7 PM.
- The Elasticsearch team will be at Berlin Buzzwords from May 25-28th.. (When we say the team, we mean most of our folks in the EU and several of our employees from the US. :)) We have many talks on the program and look forward to hosting you in the developer chill area, as well. Even better, the Berlin Elasticsearch User Group will convene a hackathon on Wednesday, May 28th. Please join us!
Norway
Martijn Laarman will be speaking on from text to full-text search at the NDC Oslo 2014 conference. The show runs from June 2-6th.
Romania
Costin Leau will speak at Topconf Bucharest 2014 on Big data real time search and analytics. Topconf Bucharest runs from June 10-13th and Costin will speak at 3:20 PM on June 12th.
Spain
Clinton Gormley has been invited to speak at the Barcelona on Rails Meetup on May 15th. Join him for a presentation on Elasticsearch's Query DSL: Not just for wizards! Doors open at 7 PM, and thanks to the fine folks at XING for hosting us!
Tunisia
David Pilato will speak at the Esprit JUG Days in Ariana. Further details of the conference schedule are forthcoming, but mark your calendar for May 7th and 8th. In the meantime, you may want to visit the conference's Facebook page.
United Kingdom
- The London Elasticsearch User Group still has a few spaces left for tonight's meetup. Doors open at 6:30 PM at Net-A-Porter's offices.
- The London Elasticsearch User Group has already announced that there next meeting will take place on June 4th. RSVP now to save your place - this one fills up very quickly!
- David Pilato will take the stage at DevoxxUK to discuss Advanced Search for Your Legacy Application DevoxxUK takes place June 12-13th in London, and David will speak on June 12th at 4 PM in Room 1.
United States
- New:Elasticsearch and MIT Sloan Data Analytics Hackathon scheduled for May 10th! Anyone can attend, not just students!
- Aaron Mildenstein will be at DevOps Days Austin on May 5th and 6th. Make sure to stop by the Elasticsearch booth to say hello!
- Rashid Khan will be presenting at Monitorama. The conference runs from May 5-7, 2014 in Portland, Oregon.
- The Elasticsearch Boston Meetup group will be getting together on May 6th at 6:00 PM. You'll hear from Igor Motov on Improving Elasticsearch Resiliency and from the Yieldbot team on their use of Elasticsearch.
- The Miami JVM Group will convene on May 13th and discuss an Introduction to Elasticsearch. Doors open at 7 PM.
- Jordan Sissel will be speaking at Gluecon 2014! Make sure to catch his talk and visit the Elasticsearch booth. The conference runs from May 21-22nd in Bloomfield, Colorado.
- The next Chicago Elasticsearch Meetup will take place on May 22nd. Doors open at 6 PM.
- Costin Leau will speak at Hadoop Summit North America on Real-time Analytics and Anomalies Detection using Elasticsearch, Hadoop and Storm. The conference runs from June 3-5th in San Jose, California. Costin will take the stage at 4:35 PM on June 3rd.
Where to Find You
Our Community Manager, Leslie Hawthorn, is hard at work to help folks create more Elasticsearch meetup groups and to help meetup organizers find more speakers. If you are interested in either effort, take a moment to let her know.
Oh yeah, we're also hiring. If you'd like us to find you for employment purposes, just drop us a note. We care more about your skill set and passion for Elasticearch, Kibana and Logstash than where you rest your head.
Trainings
If you are interested in Elasticsearch training we have courses taught by our core developers coming up in:
- Copenhagen - May 13, 2014 (core Elasticsearch training)
- Paris - May 15, 2014 (core Elasticsearch training)
- Seattle - June 3, 2014 (core Elasticsearch training)
- New York - June 4, 2014 (core Elasticsearch training)
- London - June 4, 2014 (core Elasticsearch training)
- Zurich - June 5, 2014 (core Elasticsearch training)
- San Francisco - June 6, 2014 (ELK stack hands on workshop)
- Amsterdam - June 27, 2014 (ELK stack hands on workshop)