IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

« Elasticsearch version 7.5.1 Elasticsearch version 7.4.2 »

› ›

Elasticsearch version 7.5.0

edit

Elasticsearch version 7.5.0

edit

Also see Breaking changes in 7.5.

Known issues

edit

Applying deletes or updates on an index after it has been shrunk may corrupt the index. In order to prevent this issue, it is recommended to stop shrinking read-write indices. For read-only indices, it is recommended to force-merge indices after shrinking, which significantly reduces the likeliness of this corruption in the case that deletes/updates would be applied by mistake. This bug is fixed in Elasticsearch 7.7 and later versions. More details can be found on the corresponding issue.
Stop all transforms during a rolling upgrade to 7.5. If a transform is running during upgrade, the transform audit index might disappear. (issue: #49730)
Indices created in 6.x with date and date_nanos fields using formats that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results. https://github.com/elastic/elasticsearch/pull/52555 This is fixed in Elasticsearch 7.7 and later versions.

Breaking changes

edit

Search

Add support for aliases in queries on _index. #46640 (issues: #23306, #34089)

Deprecations

edit

Aggregations

Deprecate the "index.max_adjacency_matrix_filters" setting #46394 (issue: #46324)

Allocation

Deprecate include_relocations setting #47443 (issue: #46079)

Mapping

Deprecate _field_names disabling #42854 (issue: #27239)

Search

Reject regexp queries on the _index field. #46945 (issues: #34089, #46640)

New features

edit

Features/ILM+SLM

Add API to execute SLM retention on-demand #47405 (issues: #43663, #46508)
Add retention to Snapshot Lifecycle Management #46407 (issues: #38461, #43663, #45362)

Features/Ingest

Add enrich processor #48039 (issue: #32789)

Machine Learning

Implement evaluation API for multiclass classification problem #47126 (issue: #46735)
Implement new analysis type: classification #46537 (issue: #46735)
Add audit messages for Data Frame Analytics #46521 (issue: #184)
Implement DataFrameAnalyticsAuditMessage and DataFrameAnalyticsAuditor #45967

SQL

SQL: Implement DATEDIFF function #47920 (issue: #47919)
SQL: Implement DATEADD function #47747 (issue: #47746)
SQL: Implement DATE_PART function #47206 (issue: #46372)
SQL: Add alias DATETRUNC to DATE_TRUNC function #47173 (issue: #46473)
SQL: Add PIVOT support #46489
SQL: Implement DATE_TRUNC function #46473 (issue: #46319)

Enhancements

edit

Aggregations

Adjacency_matrix aggregation memory usage optimisation. #46257 (issue: #46212)
Support geotile_grid aggregation in composite agg sources #45810 (issue: #40568)

Allocation

Do not cancel ongoing recovery for noop copy on broken node #48265 (issue: #47974)
Shrink should not touch max_retries #47719
Re-fetch shard info of primary when new node joins #47035 (issues: #42518, #46959)
Sequence number based replica allocation #46959 (issue: #46318)

Authorization

Add support to retrieve all API keys if user has privilege #47274 (issue: #46887)
Add create_doc index privilege #45806
Reducing privileges needed by built-in beats_admin role #41586

CCR

Add Pause/Resume Auto-Follower APIs to High Level REST Client #47989 (issue: #47510)
Add Pause/Resume Auto Follower APIs #47510 (issue: #46665)

CRUD

Allow optype CREATE for append-only indexing operations #47169

Cluster Coordination

Warn on slow metadata persistence #47005
Improve LeaderCheck rejection messages #46998

Engine

Do not warm up searcher in engine constructor #48605 (issue: #47186)
Refresh should not acquire readLock #48414 (issue: #47186)
Avoid unneeded refresh with concurrent realtime gets #47895
sync before trimUnreferencedReaders to improve index preformance #47790 (issues: #46201, #46203)
Limit number of retaining translog files for peer recovery #47414
Remove isRecovering method from Engine #47039

Features/ILM+SLM

Separate SLM stop/start/status API from ILM #47710 (issue: #43663)
Set default SLM retention invocation time #47604 (issue: #43663)
ILM: Skip rolling indexes that are already rolled #47324 (issue: #44175)
Add support for POST requests to SLM Execute API #47061
Wait for snapshot completion in SLM snapshot invocation #47051 (issues: #38461, #43663)
Add node setting for disabling SLM #46794 (issue: #38461)
ILM: parse origination date from index name #46755 (issues: #42449, #46561)
[ILM] Add date setting to calculate index age #46561 (issue: #42449)

Features/Ingest

Add the ability to require an ingest pipeline #46847

Features/Java High Level REST Client

add function submitDeleteByQueryTask in class RestHighLevelClient #46833
return Cancellable in RestHighLevelClient #45688 (issue: #44802)

Features/Java Low Level REST Client

Add cloudId builder to the HLRC #47868
Add support for cancelling async requests in low-level REST client #45379 (issues: #43332, #44802)

Features/Monitoring

Remove hard coded version_created in default monitoring alerts #47744

Infra/Circuit Breakers

Emit log message when parent circuit breaker trips #47000
Fix G1 GC default IHOP #46169

Infra/Core

Introduce system JVM options #48252 (issue: #48222)
Set start of the week to Monday for root locale #43652 (issues: #41670, #42588, #43275)

Infra/Packaging

Package the JDK into jdk.app on macOS #48765
Move ES_TMPDIR substitution into jvm options parser #47189 (issue: #47133)
Clarify missing java error message #46160 (issue: #44139)

Infra/Scripting

Add explanations to script score queries #46693

Infra/Settings

Do not reference values for filtered settings #48066
Allow setting validation against arbitrary types #47264 (issue: #25560)
Clarify error message on keystore write permissions #46321
Add more meaningful keystore version mismatch errors #46291 (issue: #44624)

Machine Learning

Throw an exception when memory usage estimation endpoint encounters empty data frame. #49143 (issue: #49140)
Change format of MulticlassConfusionMatrix result to be more self-explanatory #48174 (issue: #46735)
Make num_top_classes parameter’s default value equal to 2 #48119 (issue: #46735)
[ML] Add option to stop datafeed that finds no data #47922
Allow integer types for classification’s dependent variable #47902 (issue: #46735)
[ML] Add lazy assignment job config option #47726
[ML] Additional outlier detection parameters #47600
[ML] More accurate job memory overhead #47516
[ML] Throttle the delete-by-query of expired results #47177 (issues: #47003, #47103)

Mapping

Add migration tool checks for _field_names disabling #46972 (issues: #42854, #46681)

Network

Introduce simple remote connection strategy #47480
Enhanced logging when transport is misconfigured to talk to HTTP port #45964 (issue: #32688)

Recovery

Do not send recovery requests with CancellableThreads #46287 (issue: #46178)

SQL

SQL: make date/datetime and interval types compatible in conditional functions #47595 (issue: #46674)
SQL: use calendar interval of 1y instead of fixed interval for grouping by YEAR and HISTOGRAMs #47558 (issue: #40162)
SQL: Support queries with HAVING over SELECT #46709 (issue: #37051)
SQL: Add support for shape type #46464 (issues: #43644, #46412)

Search

Remove response search phase from ExpandSearchPhase #48401
Add builder for distance_feature to QueryBuilders #47846 (issue: #47767)
Fold InitialSearchPhase into AbstractSearchAsyncAction #47182
max_children exist only in top level nested sort #46731
First round of optimizations for vector functions. #46294 (issues: #45390, #45936, #46103, #46155, #46190, #46202)
Throw exception in scroll requests using from #46087 (issues: #26235, #44493, #9373)

Snapshot/Restore

Track Repository Gen. in BlobStoreRepository #48944 (issues: #38941, #47520, #47834, #49048)
Resume partial download from S3 on connection drop #46589
More Efficient Ordering of Shard Upload Execution #42791

Transform

[ML][Transforms] allow executor to call start on started task #46347
[ML-DataFrame] improve error message for timeout case in stop #46131 (issue: #45610)
[ML][Data Frame] add support for wait_for_checkpoint flag on _stop API #45469 (issue: #45293)

Bug fixes

edit

Aggregations

Fix ignoring missing values in min/max aggregations #48970 (issue: #48905)
DocValueFormat implementation for date range fields #47472 (issues: #47323, #47469)

Allocation

Auto-expand replicated closed indices #48973
Handle negative free disk space in deciders #48392 (issue: #48380)
Dangling indices strip aliases #47581
Cancel recoveries even if all shards assigned #46520
Fail allocation of new primaries in empty cluster #43284 (issue: #41073)

Analysis

Reset Token position on reuse in predicate_token_filter #47424 (issue: #47197)

Audit

Audit log filter and marker #45456 (issue: #47251)

Authentication

Add owner flag parameter to the rest spec #48500 (issue: #48499)
Add populate_user_metadata in OIDC realm #48357 (issue: #48217)
Remove unnecessary details logged for OIDC #48271
Fix AD realm additional metadata #47179 (issue: #45848)
Fallback to realm authc if ApiKey fails #46538
PKI realm accept only verified certificates #45590

Authorization

Fix security origin for TokenService#findActiveTokensFor… #47418 (issue: #47151)
Use should clause instead of filter when querying native privileges #47019
Do not rewrite aliases on remove-index from aliases requests #46989
Validate index and cluster privilege names when creating a role #46361 (issue: #29703)
Validate query field when creating roles #46275 (issue: #34252)

CCR

CCR should auto-retry rejected execution exceptions #49213
Do not auto-follow closed indices #47721 (issue: #47582)
Relax maxSeqNoOfUpdates assertion in FollowingEngine #47188 (issue: #47137)
Handle lower retaining seqno retention lease error #46420 (issues: #46013, #46416)

CRUD

Close query cache on index service creation failure #48230 (issue: #48186)
Use optype CREATE for single auto-id index requests #47353
Ignore replication for noop updates #46458 (issues: #41065, #44603, #46366)

Client

Correct default refresh policy for security APIs #46896

Cluster Coordination

Ignore metadata of deleted indices at start #48918
Omit writing index metadata for non-replicated closed indices on data-only node #47285 (issue: #47276)
Assert no exceptions during state application #47090 (issue: #47038)
Remove trailing comma from nodes lists #46484

Distributed

Closed shard should never open new engine #47186 (issues: #45263, #47060)
Fix false positive out of sync warning in synced-flush #46576 (issues: #28464, #30244)
Suppress warning logs from background sync on relocated primary #46247 (issues: #40800, #42241)

Engine

Greedily advance safe commit on new global checkpoint #48559 (issue: #48532)

Features/ILM+SLM

Don’t halt policy execution on policy trigger exception #49128
Don’t schedule SLM jobs when services have been stopped #48658 (issue: #47749)
Ensure SLM stats does not block an in-place upgrade from 7.4 #48367
Ensure SLM stats does not block an in-place upgrade from 7.4 #48361
Add SLM support to xpack usage and info APIs #48096 (issue: #43663)
Change policy_id to list type in slm.get_lifecycle #47766 (issue: #47765)
Throw error retrieving non-existent SLM policy #47679 (issue: #47664)
Handle partial failure retrieving segments in SegmentCountStep #46556
Fixes for API specification #46522

Features/Indices APIs

Fix Rollover error when alias has closed indices #47148 (issue: #47146)

Features/Ingest

Do not wrap ingest processor exception with IAE #48816 (issue: #48810)
Introduce dedicated ingest processor exception #48810 (issue: #48803)
Allow dropping documents with auto-generated ID #46773 (issue: #46678)
Expose cache setting in UserAgentPlugin #46533

Features/Java High Level REST Client

fix incorrect comparison #48208
Fix ILM HLRC Javadoc→Documentation links #48083
Change HLRC count request to accept a QueryBuilder #46904 (issue: #46829)
[HLRC] Send min_score as query string parameter to the count API #46829 (issue: #46474)
HLRC multisearchTemplate forgot params #46492 (issue: #46488)
Added fields for MultiTermVectors (#42232) #42877 (issue: #42232)

Features/Java Low Level REST Client

Update http-core and http-client dependencies #46549 (issues: #45379, #45577, #45808)

Features/Monitoring

[Monitoring] Add new cluster privilege now necessary for the stack monitoring ui #47871
Validating monitoring hosts setting while parsing #47246 (issue: #47125)

Features/Watcher

Fix class used to initialize logger in Watcher #46467
Fix wrong URL encoding in watcher HTTP client #45894 (issue: #44970)
Prevent deadlock by using separate schedulers #48697 (issues: #41451, #47599)
Fix cluster alert for watcher/monitoring IndexOutOfBoundsExcep… #45308 (issue: #43184)

Geo

Geo: implement proper handling of out of bounds geo points #47734 (issue: #43916)
Geo: Fixes indexing of linestrings that go around the globe #47471 (issues: #43826, #43837)
Provide better error when updating geo_shape field mapper settings #47281 (issue: #47006)
Geo: fix indexing of west to east linestrings crossing the antimeridian #46601 (issue: #43775)
Reset queryGeometry in ShapeQueryTests #45974 (issue: #45628)

Highlighting

Fix highlighting of overlapping terms in the unified highlighter #47227
Fix highlighting for script_score query #46507 (issue: #46471)

Infra/Core

Don’t drop user’s MaxDirectMemorySize flag on jdk8/windows #48657 (issues: #44174, #48365)
Warn when MaxDirectMemorySize may be incorrect (Windows/JDK8 only issue) #48365 (issue: #47384)
Support optional parsers in any order with DateMathParser and roundup #46654 (issue: #45284)

Infra/Logging

SearchSlowLog uses a non thread-safe object to escape json #48363 (issues: #44642, #48358)

Infra/Scripting

Drop stored scripts with the old style-id #48078 (issue: #47593)

Machine Learning

[ML] Fixes for stop datafeed edge cases #49191 (issues: #43670, #48931)
[ML] Avoid NPE when node load is calculated on job assignment #49186 (issue: #49150)
Do not throw exceptions resulting from persisting datafeed timing stats. #49044 (issue: #49032)
[ML] Deduplicate multi-fields for data frame analytics #48799 (issues: #48756, #48770)
[ML] Prevent fetching multi-field from source #48770 (issue: #48756)
[ML] Fix detection of syslog-like timestamp in find_file_structure #47970
Fix serialization of evaluation response. #47557
[ML] Reinstate ML daily maintenance actions #47103 (issue: #47003)
[ML] fix two datafeed flush lockup bugs #46982

Network

Fix es.http.cname_in_publish_address Deprecation Logging #47451 (issue: #47436)

Recovery

Ignore Lucene index in peer recovery if translog corrupted #49114

Reindex

Fix issues with serializing BulkByScrollResponse #45357

SQL

SQL: Fix issue with mins & hours for DATEDIFF #49252
SQL: Fix issue with negative literels and parentheses #48113 (issue: #48009)
SQL: add "format" for "full" date range queries #48073 (issue: #48033)
SQL: Fix arg verification for DateAddProcessor #48041
SQL: Fix Nullability of DATEADD #47921
SQL: Allow whitespaces in escape patterns #47577 (issue: #47401)
SQL: fix multi full-text functions usage with aggregate functions #47444 (issue: #47365)
SQL: Check case where the pivot limit is reached #47121 (issue: #47002)
SQL: Properly handle indices with no/empty mapping #46775 (issue: #46757)
SQL: improve ResultSet behavior when no rows are available #46753 (issue: #46750)
SQL: use the correct data type for types conversion #46574 (issue: #46090)
SQL: Fix issue with common type resolution #46565 (issue: #46551)
SQL: fix scripting for grouped by datetime functions #46421 (issue: #40241)
SQL: Use null schema response #46386 (issue: #46381)
SQL: Fix issue with IIF function when condition folds #46290 (issue: #46268)
SQL: Fix issue with DataType for CASE with NULL #46173 (issue: #46032)
SQL: Failing Group By queries due to different ExpressionIds #43072 (issues: #33361, #34543, #36074, #37044, #40001, #40240, #41159, #42041, #46316)
SQL: wrong number of values for columns #42122

Search

Lucene#asSequentialBits gets the leadCost backwards. #48335
Ensure that we don’t call listener twice when detecting a partial failures in _search #47694
Fix alias field resolution in match query #47369
Multi-get requests should wait for search active #46283 (issue: #27500)
Resolve the incorrect scroll_current when delete or close index #45226
Don’t apply the plugin’s reader wrapper in can_match phase #47816 (issue: #46817)

Security

Remove uniqueness constraint for API key name and make it optional #47549 (issue: #46646)
Initialize document subset bit set cache used for DLS #46211 (issue: #45147)

Snapshot/Restore

Fix RepoCleanup not Removed on Master-Failover #49217
Make FsBlobContainer Listing Resilient to Concurrent Modifications #49142 (issue: #37581)
Fix SnapshotShardStatus Reporting for Failed Shard #48556 (issue: #48526)
Cleanup Concurrent RepositoryData Loading #48329 (issue: #48122)
Fix Bug in Azure Repo Exception Handling #47968
Make loadShardSnapshot Exceptions Consistent #47728 (issue: #47507)
Fix Snapshot Corruption in Edge Case #47552 (issues: #46250, #47550)
Fix Bug in Snapshot Status Response Timestamps #46919 (issue: #46913)
Normalize Blob Store Repo Paths #46869 (issue: #41814)
GCS deleteBlobsIgnoringIfNotExists should catch StorageException #46832 (issue: #46772)
Execute SnapshotsService Error Callback on Generic Thread #46277
Make Snapshot Logic Write Metadata after Segments #45689 (issue: #41581)

Store

Allow truncation of clean translog #47866

Task Management

Fix .tasks index strict mapping: parent_id should be parent_task_id #48393

Transform

[Transform] do not fail checkpoint creation due to global checkpoint mismatch #48423 (issue: #48379)
[7.5][Transform] prevent assignment if any node is older than 7.4 #48055 (issue: #48019)
[Transform] prevent assignment to nodes older than 7.4 #48044 (issue: #48019)
[ML][Transforms] fix bwc serialization with 7.3 #48021
[ML][Transforms] signal listener early on task _stop failure #47954
[ML][Transform] Use field_caps API for mapping deduction #46703 (issue: #46694)
[ML-DataFrame] Fix off-by-one error in checkpoint operations_behind #46235