This Week in Elasticsearch and Apache Lucene - Cluster Cloning in Hosted Elasticsearch

Welcome to This Week in Elasticsearch and Apache Lucene! With this weekly series, we're bringing you an update on all things Elasticsearch and Apache Lucene at Elastic, including the latest on commits, releases and other learning resources.

Top News

Just how easy is it clone your cluster in our hosted #Elasticsearch service? (Hint: very) https://t.co/I3ySOOFz04 pic.twitter.com/M9HoK1KTgw
— elastic (@elastic) January 29, 2016

Elasticsearch Core

Changes in 2.2:

• Closures are once again allowed in the Groovy scripting plugin, and a PR has been submitted to Groovy to remove the need for the supressAccessChecks permission.
• Translog recovery is fast again.
• Command line options now work correctly on Windows.
• The Tribe node wasn't passing on a custom --path.conf to its node clients, which resulted in security exceptions.
• Query and top-level inner hits results shouldn't overwrite each other.
• Geo-shapes didn't work in the percolator if map_unmapped_fields_as_string was enabled.

Changes in 2.x:

• Disabled fielddata loading was silently ignored on empty indices.
• Throw an exception if the Lucene version is not the expected one.
• Include the exception name for not serializable exceptions.

Changes in master:

• Shard failure requests for no longer existing shards should always be considered successful.
• The term-level fuzzy query is deprecated in favour of the match query with the fuzziness parameter.
• Boolean settings in mappings are now strict.
• The "index" mapping param now accepts only true/false.
• Setting index: false will no longer disable doc values as well.
• Doc values are controlled by the doc_values setting only, not by fielddata_format.
• Many more global settings have been migrated to the new settings infrastructure.
• The new scripting language is called Painless.
• Deep pagination is now possible with the search_after parameter.
• The ingest node should make deep copies of data structures.
• Disabled the ability to fsync on every operation (instead of every request) and only schedule fsync if really needed.
• Shards are marked as active during recovery, to ensure the indexing buffer is big enough.
• TermVector APIs no longer update mappings.
• Tracking of parent tasks now include master node, replication, and broadcast actions.
• Load average info has been normalised across different OSes.
• Improve exceptions from ingest pipelines.
• Ensure all resources are closed when closing a node.

Ongoing:

• Work has started on using UUIDs to identify indices on the file system, rather than relying just on index names.
• The reindex API is starting to use the task management framework.
• Search refactoring continues with the suggesters and sorting. A design bug in aggs refactoring (creating one instance per node instead of per shard) will require quite a big change.
• Updating mappings with update_all_types isn't working correctly.

Apache Lucene

Watch This Space

Stay tuned to this blog, where we'll share more news on the whole Elastic ecosystem including news, learning resources and cool use cases!