This Week in Elasticsearch - 2020-02-21

By
Tom Callahan
•
Alison Goryachev
•
Zachary Tong
•
Yannick Welsch

Elasticsearch

Feature parity between Prefix Trees and BKD-backed geo_shapes

There are currently a few limitations when querying geo_shapes that have been indexed using BKD-backed strategy. In particular, users are prevented to query this data  multi_point and circle shapes. We are currently working in eliminating this limitation:

  • Multi-point queries: a PR has been merged in Elasticsearch that removes the limitation of BKD-backed geo_shapes when querying using multi-points.
  • Distance queries: a PR has been merged in Lucene that adds distance query capabilities to LatLonShape and XYShape.

While many users have migrated their geospatial data to the new BKD trees format, the lack of feature parity has held back some users. Adding multi-point queries to our BKD tree should unlock a number of users who were holding off upgrading. It's flu season, so a topical example would be a multi-point shape that represents each case of this year's flu infection. You could query that shape against an index of country shapes and find all the countries that intersect with the shape and have active flu outbreaks. The same idea applies to any "point of interest" query where you have multiple discete points that are categorized together, like cafes in a city, or national parks across states, etc.

Distance queries are basically circle queries: find all values within 10km of a center point. This is a super common operation that you'll find on any geospatial application (find all cafe's in 5 mile radius, etc). It was previously restricted to points (like cafes), but with shape support you can ask questions like "Find all state boundaries that are within 100 miles". And since shapes can also represent lines, you can formulate questions like "Find all shipping paths that passed within 20 miles of this point".

Painless Lab

We're very proud share early work on a featuer we're calling "Painless Lab" and, as the name implies, this is an app for experimenting with Painless! Users who become more proficient with Painless will become more proficient where Painless is used, such as Watcher, ingest node pipelines, and scripted fields. If you'd like to take the Painless Lab for a spin, check out the feature branch.

Painless Lab grew out of an internal experiement, and recently we've worked to improve the experience. These improvements include giving the user immediate feedback by updating the output as they type, creating a larger code editing area for typing the Painless script, grouping secondary inputs like parameters and custom document data under a single set of tabs, and adding copious links to documentation. We've also added a default script to help the user get started. It renders a smiley face - check out the screenshot!

Painless Lab rendering smiley face

S3 Repository Compatibility Improvements

We added region and signer algorithm overrides for S3 repositories that provides wider compatibility with third party S3 implementations such as Ceph, as well as a wider range of Minio configurations. Note that compatibility with a particular repository implementation does not imply full support.

Changes

Breaking Changes in Rally

Breaking Changes in 1.4.1:

  • Use zeros instead of whitespaces as padding bytes #899