This Week in Elasticsearch and Apache Lucene: Developer All Hands meeting

By
Ryan Ernst

Welcome to This Week in Elasticsearch and Apache Lucene! With this new weekly post, we're bringing you an update on all things Elasticsearch and Apache Lucene at Elastic, including the latest on commits, releases and other learning resources.

We're experimenting with this new format, and we welcome your feedback. Give us a shout on Twitter to let us know what you think!

Developer All Hands Meeting

Last weekend, the Elasticsearch developer team met for an all hands meeting following Elastic{ON}. This was the first time since June of last year that the entire engineering team came together in one place. We're an incredibly distributed team that's doubled in size since then, so coming together to get on the same page, in the same place was, in a word, fantastic. (P.S. We're hiring.)

We took the time to learn about our new friends at Found, discuss the immediate and long term future, and just talk shop about the projects we love.  But we didn't stop there: we discussed our culture, our communication styles, and of course, we also got to know each other better in a way one can only do in person: over a beer or two. We even spent a night hacking together which produced some pretty cool ideas.

Top News

In case you missed it, we released Elasticsearch 1.5.0 on Monday! This is the latest stable version of Elasticsearch. It contains a number of important resiliency enhancements and bug fixes, and we advise all users to upgrade. This release also includes two new experimental features, inner hits and shadow replicas.

Read all about the release and these new features in Clinton Gormley's release announcement.

Elasticsearch: Level Up

We'll share the wider world of all Elasticsearch news with you weekly on Wednesdays in our This Week in Elastic post. But we'll always bring you at least one awesome learning resource here weekly too.

From our new Found family members: Designing for your multi-tenant Elasticsearch application, including tips and tricks for scaling up and out.

Have a recommended article, blog post or video you think we should feature in this section? Let us know!

Elasticsearch Core

  • Core: Deprecated the thrift and memcached transports (#10167, 1.6.0, 1.5.0)
  • Core: Remove references to the thrift and memcached transport plugins (#10169, 2.0.0)
  • Replication: Remove `async` replication (#10171, 2.0.0)
  • Internal: Add before/afterIndexShardDelete callback (#10173, 2.0.0, 1.5.0)
  • Snapshots: increase default rate limiting for snapshot, restore and recovery to 40 MB/sec (#10185, 2.0.0, 1.6.0, 1.5.0)
  • Java API: remove duplicated consistency level and replication type setters (#10188, 2.0.0, 1.6.0)
  • Core: Remove benchmark api (#10184, 2.0.0)
  • Internal: Schedule transport ping interval (#10189, 2.0.0, 1.6.0)
  • Removed duplicate timeout param (#10205, 2.0.0, 1.6.0, 1.5.0, 1.4.5, 1.3.10)
  • Internal: Upgrade to Jackson 2.5.1 (#10210, 2.0.0, 1.6.0)
  • Internal: Fix string comparisons (#10204, 2.0.0, 1.6.0)
  • Allocation: Verify shards index UUID when fetching started shards (#10200, 2.0.0)
  • Recovery: only cancel when primary completed relocation (#10218, 2.0.0, 1.6.0)
  • Transport: fix racing condition in timeout handling (#10220, 2.0.0, 1.6.0, 1.5.1)
  • Docs: clearer MLT documentation (#10191, 1.6.0, 1.5.1)
  • Geo: fix docs for geo_point "validate" option (#10168, 2.0.0, 1.5.1)
  • Internal: get tests working on java 9 (#10228, 2.0.0, 1.6.0, 1.5.1)
  • Geo: Fix validate_merge policy for GeoPointFieldMapper (#10165, 2.0.0, 1.5.1)
  • API: Deprecate delete-by-query in client/transport/action APIs too (#10239, 1.6.0, 1.5.1)
  • Mappings: Remove delete mapping API (#10231, 2.0.0)
  • API: changed all the links in json specs to the new site (#10240, 2.0.0, 1.6.0)
  • Scripts: Added "lang" field to documentation of script score definition (#10243, 2.0.0, 1.6.0, 1.5.1)
  • Stats: add node id to _cat apis (#10246, 2.0.0, 1.6.0)
  • Recovery: add total operations to the `_recovery` API (#10042, 2.0.0, 1.5.0)
  • Transport: Return useful error message on potential HTTP connect (#10108, 2.0.0, 1.6.0)
  • Plugins: Always send current ES version when downloading plugins (#10131, 2.0.0, 1.6.0, 1.5.0)
  • Core: fix typo when primary is not available to index a document (UnavailableShardsException) (#10140, 2.0.0, 1.5.0)
  • Tests: use bigger back compat indexes (#10143, 2.0.0, 1.6.0)
  • Http: add option to only return simple exception messages (#10117, 2.0.0, 1.5.0)
  • Tests: Add tests that triggers truncated files on recovery (#10155, 2.0.0, 1.6.0, 1.5.0)
  • Docs: Need additional line in config file, to work tutorial's example. (#10074, 2.0.0, 1.6.0, 1.5.0, 1.4.5)
  • Suggest: return an HTTP error code when a suggest request failed instead of 200 (#10104, 2.0.0, 1.6.0, 1.5.0)
  • Internal: Move GatewayShardsState logic into IndexShard (#10093, 2.0.0)
  • Delete api: remove broadcast delete if routing is missing when required (#10136, 2.0.0)
  • Recovery: add throttle stats (#10097, 2.0.0, 1.5.0)
  • Cleanup: Remove unsafe field in BytesStreamInput (#10157, 2.0.0, 1.6.0)
  • Transport: Close all resources if doStart fails (#9898, 2.0.0, 1.6.0)
  • _cat/nodes: Thread null handling through stats and info (#9938, 2.0.0, 1.6.0)
  • Transport: fix potential NPE in new tracer log if request timeout (#9994, 2.0.0, 1.6.0, 1.5.0)
  • Internal: Better detection of CBOR (#10026, 2.0.0, 1.6.0)
  • Core: fix handling of IndicesOptions in update settings REST API (#10030, 2.0.0, 1.6.0)
  • Function score: undo "Remove explanation of query score from functions" (#9826, 2.0.0, 1.6.0, 1.5.1)
  • Percolator: Take filters from aliases into account (#9590, 2.0.0, 1.5.0)
  • Recovery: node shut down during the last phase of recovery needlessly fails shard (#9496, 2.0.0, 1.5.0)
  • Logging: Change logging to warning to match pattern (#9593, 2.0.0, 1.6.0)
  • Geo: "validate" option for type "geo_point" in mapping is not loading (#9539, 2.0.0, 1.6.0)
  • Stats: Add node name to _cat/recovery output (#9353, 2.0.0, 1.6.0)
  • Allocation: Early terminate if the cluster can't be rebalanced (#9162, 2.0.0, 1.6.0)
  • Internal: Don't broadcast deletes to all shards (#9123, 2.0.0)
  • API: Remove indices_boost URL param, closes #6281 (#9244, 2.0.0, 1.6.0)
  • Core: use Lucene's defaults for compound file format (#8934, 2.0.0, 1.5.0)
  • Resiliency: Throw exception if the JVM will corrupt data. (#7580, 2.0.0, 1.5.0, 1.4.5)

Elasticsearch Plugin Releases

Elastic and Apache Lucene

Watch This Space

Stay tuned to the blog, where we'll share more news on the whole ELK ecosystem including news, learning resources and cool use cases!