Elasticsearch highlights

edit

This list summarizes the most important enhancements in Elasticsearch 7.16. For the complete list, go to Elasticsearch release highlights.

SQL: Support for cross-cluster search

edit

Elasticsearch SQL now supports cross-cluster search (CCS) using the <remote_cluster>:<target> syntax, where <remote_cluster> maps to a SQL catalog (cluster) and <target> to a table (index or data stream).

Search: Improved can-match phase for scalability

edit

If a search hits a large number of shards, the search operation includes a pre-filter phase called the can-match phase. During this phase, Elasticsearch checks if an impacted shard contains data that could potentially match the search query. If not, Elasticsearch doesn’t run the query on the shard.

Previously, the search’s coordinating node sent an individual request to each shard checked during the can-match phase. However, if the search needed to check thousands of shards, the coordinating node would need to handle thousands of requests, resulting in high overhead.

With 7.16, the coordinating node instead sends a single request to each data node during the can-match phase. This request covers can-match checks for all impacted shards on the node, significantly reducing the number of requests and related overhead.

Field capabilities: Results gathered by node for scalability

edit

Previously, field caps were gathered by sending a request per index to the appropriate nodes in the cluster. This resulted in high overhead when many indices were targeted. Now requests that target indices on a single node are grouped together, resulting in no more than a single request per node.

Frozen tier: Cached requests and queries

edit

Requests and queries against indices in the frozen tier are now cached, which improves performance for subsequent requests and queries.

Enrich processor: New range enrich policy type

edit

With 7.16, we added the range enrich policy type for the enrich processor. You can use a range policy to enrich incoming documents based on a number, date, or IP address that matches a range in the enrich index.

For example, if incoming documents contain an IP address, you can use a range policy to enrich the documents based on their IP range. For an in-depth example, see Example: Enrich your data by matching a value to a range.

Data streams: Segment sorting for faster searches

edit

For data streams, Elasticsearch now sorts a backing index’s segments by maximum @timestamp value in descending order. This speeds up searches sorted by @timestamp, which are common for data streams.

EQL: with runs statements for repeated events

edit

In 7.16, we added the with runs statement syntax to EQL sequence queries. Sometimes you want to find a sequence that contains an event multiple times in succession. Rather than type the same event criteria multiple times, you can use a with runs statement to declare the criteria once and run it successively. For more details, check out the EQL syntax documentation.