A newer version is available. For the latest information, see the current release documentation.
« Elasticsearch version 7.0.0-beta1 Elasticsearch version 7.0.0-alpha1 »
Elastic Docs ›Elasticsearch Guide [7.17] ›Release notes

Elasticsearch version 7.0.0-alpha2

edit

Elasticsearch version 7.0.0-alpha2

edit

Breaking changes

edit
Authentication
Circuit Breakers
  • Lower fielddata circuit breaker’s default limit #27162 (issue: #27130)
CCR
  • Change get autofollow patterns API response format #36203 (issue: #36049)
Index APIs
Ranking
  • Forbid negative scores in function_score query #35709 (issue: #33309)
Scripting
  • Delete deprecated getValues from ScriptDocValues #36183 (issue: #22919)
Search
  • Remove the deprecated _termvector endpoint. #36131 (issues: #36098, #8484)
  • Remove deprecated Graph endpoints #35956
  • Validate metadata on _msearch #35938 (issue: #35869)
  • Make hits.total an object in the search response #35849 (issue: #33028)
  • Remove the distinction between query and filter context in QueryBuilders #35354 (issue: #35293)
  • Throw a parsing exception when boost is set in span_or query (#28390) #34112 (issue: #28390)
ZenDiscovery
  • Best-effort cluster formation if unconfigured #36215

Breaking Java changes

edit
ZenDiscovery
  • Make node field in JoinRequest private #36405

Deprecations

edit
Core
  • Deprecate use of scientific notation in epoch time parsing #36691
  • Add backcompat for joda time formats #36531
Machine Learning
  • Deprecate X-Pack centric ML endpoints #36315 (issue: #35958)
Mapping
  • Deprecate types in index API #36575 (issues: #35190, #35790)
  • Deprecate uses of _type as a field name in queries #36503 (issue: #35190)
  • Deprecate types in update_by_query and delete_by_query #36365 (issue: #35190)
  • For msearch templates, make sure to use the right name for deprecation logging. #36344
  • Deprecate types in termvector and mtermvector requests. #36182
  • Deprecate types in update requests. #36181
  • Deprecate types in document delete requests. #36087
  • Deprecate types in get, exists, and multi get. #35930
  • Deprecate types in search and multi search templates. #35669
  • Deprecate types in explain requests. #35611
  • Deprecate types in validate query requests. #35575
  • Deprecate types in count and msearch. #35421 (issue: #34041)
Migration
  • Deprecate X-Pack centric Migration endpoints #35976 (issue: #35958)
Monitoring
  • Deprecate /_xpack/monitoring/* in favor of /_monitoring/* #36130 (issue: #35958)
Rollup
  • Re-deprecate xpack rollup endpoints #36451 (issue: #36044)
  • Deprecate X-Pack centric rollup endpoints #35962 (issue: #35958)
Scripting
  • Adds deprecation logging to ScriptDocValues#getValues. #34279 (issue: #22919)
  • Conditionally use java time api in scripting #31441
Search
  • Remove X-Pack centric graph endpoints #36010 (issue: #35958)
Security
  • Deprecate X-Pack centric license endpoints #35959 (issue: #35958)
  • Deprecate /_xpack/security/* in favor of /_security/* #36293 (issue: #35958)
SQL
  • Deprecate X-Pack SQL translate endpoint #36030
  • Deprecate X-Pack centric SQL endpoints #35964 (issue: #35958)
Watcher
  • Deprecate X-Pack centric watcher endpoints #36218 (issue: #35958)

New features

edit
Analysis
  • Add support for inlined user dictionary in Nori #36123 (issue: #35842)
  • Add a prebuilt ICU Analyzer #34958 (issue: #34285)
Java High Level REST Client
Java Low Level REST Client
  • Make warning behavior pluggable per request #36345
  • Add PreferHasAttributeNodeSelector #36005
Geo
  • Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default geo_shape indexing approach #36751 (issue: #35320)
  • Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default geo_shape indexing approach #35320 (issue: #32039)
Machine Learning
  • Add delayed datacheck to the datafeed job runner #35387 (issue: #35131)
Mapping
  • Make typeless APIs usable with indices whose type name is different from _doc #35790 (issue: #35190)
SQL
Search

Enhancements

edit
Aggregations
  • Added keyed response to pipeline percentile aggregations 22302 #36392 (issue: #22302)
  • Enforce max_buckets limit only in the final reduction phase #36152 (issues: #32125, #35921)
  • Histogram aggs: add empty buckets only in the final reduce step #35921
  • Handles exists query in composite aggs #35758
  • Added parent validation for auto date histogram #35670
Analysis
  • Allow word_delimiter_graph_filter to not adjust internal offsets #36699 (issues: #33710, #34741)
  • Ensure TokenFilters only produce single tokens when parsing synonyms #34331 (issue: #34298)
Audit
  • Add "request.id" to file audit logs #35536
Authentication
  • Invalidate Token API enhancements - HLRC #36362 (issue: #35388)
  • Add DEBUG/TRACE logs for LDAP bind #36028
  • Add Tests for findSamlRealm #35905
  • Add realm information for Authenticate API #35648
  • Formal support for "password_hash" in Put User #35242 (issue: #34729)
Authorization
  • Improve exact index matching performance #36017
  • manage_token privilege for kibana_system #35751
  • Grant .tasks access to kibana_system role #35573
Build
  • Sounds like typo in exception message #35458
  • Allow set section in setup section of REST tests #34678
CCR
  • Add time since last auto follow fetch to auto follow stats #36542 (issues: #33007, #35895)
  • Clean followed leader index UUIDs in auto follow metadata #36408 (issue: #33007)
  • Change AutofollowCoordinator to use wait_for_metadata_version #36264 (issues: #33007, #35895)
  • Refactor AutoFollowCoordinator to track leader indices per remote cluster #36031 (issues: #33007, #35895)
Core
  • Override the JVM DNS cache policy #36570
  • Replace usages of AtomicBoolean based block of code by the RunOnce class #35553 (issue: #35489)
  • Added wait_for_metadata_version parameter to cluster state api. #35535
  • Extract RunOnce into a dedicated class #35489
  • Introduce elasticsearch-core jar #28191 (issue: #27933)
  • Rename core module to server #28180 (issue: #27933)
CRUD
  • Rename seq# powered optimistic concurrency control parameters to ifSeqNo/ifPrimaryTerm #36757 (issues: #10708, #36148)
  • Expose Sequence Number based Optimistic Concurrency Control in the rest layer #36721 (issues: #10708, #36148)
  • Add doc’s sequence number + primary term to GetResult and use it for updates #36680 (issues: #10708, #36148)
  • Add seq no powered optimistic locking support to the index and delete transport actions #36619 (issues: #10708, #36148)
Distributed
  • [Close Index API] Mark shard copy as stale if needed during shard verification #36755
  • [Close Index API] Refactor MetadataIndexStateService #36354 (issue: #36249)
  • [Close Index API] Add TransportShardCloseAction for pre-closing verifications #36249
  • TransportResyncReplicationAction should not honour blocks #35795 (issues: #35332, #35597)
  • Expose all permits acquisition in IndexShard and TransportReplicationAction #35540 (issue: #33888)
  • [RCI] Check blocks while having index shard permit in TransportReplicationAction #35332 (issue: #33888)
Engine
  • Add sequence numbers based optimistic concurrency control support to Engine #36467 (issues: #10708, #36148)
  • Require soft-deletes when access changes snapshot #36446
  • Use delCount of SegmentInfos to calculate numDocs #36323
  • Always configure soft-deletes field of IndexWriterConfig #36196 (issue: #36141)
  • Enable soft-deletes by default on 7.0.0 or later #36141
  • Always return false from refreshNeeded on ReadOnlyEngine #35837 (issue: #35785)
  • Add a _freeze / _unfreeze API #35592 (issue: #34352)
  • [RCI] Add IndexShardOperationPermits.asyncBlockOperations(ActionListener<Releasable>) #34902 (issue: #33888)
Features
  • Simplify deprecation issue levels #36326
Index APIs
Ingest
  • Grok fix duplicate patterns JAVACLASS and JAVAFILE #35886
  • Implement Drop Processor #32278 (issue: #23726)
Java High Level REST Client
Java Low Level REST Client
  • On retry timeout add root exception #25576
Monitoring
Geo
  • Adds a name of the field to geopoint parsing errors #36529 (issue: #15965)
  • Add support to ShapeBuilders for building Lucene geometry #35707 (issue: #35320)
  • Add ST_WktToSQL function #35416 (issue: #29872)
License
  • Require acknowledgement to start_trial license #30135 (issue: #30134)
Machine Learning
  • Create the ML annotations index #36731 (issues: #26034, #33376)
  • Split in batches and migrate all jobs and datafeeds #36716 (issue: #32905)
  • Add cluster setting to enable/disable config migration #36700 (issue: #32905)
  • Add audits when deprecation warnings occur with datafeed start #36233
  • Add lazy parsing for DatafeedConfig:Aggs,Query #36117
  • Add support for lazy nodes (#29991) #34538 (issue: #29991)
Network
  • Unify transport settings naming #36623
  • Add sni name to SSLEngine in netty transport #33144 (issue: #32517)
  • Add cors support to NioHttpServerTransport #30827 (issue: #28898)
  • Reintroduce mandatory http pipelining support #30820
  • Make http pipelining support mandatory #30695 (issues: #28898, #29500)
  • Add nio http server transport #29587 (issue: #28898)
  • Selectors operate on channel contexts #28468 (issue: #27260)
  • Unify nio read / write channel contexts #28160 (issue: #27260)
  • Create nio-transport plugin for NioTransport #27949 (issue: #27260)
  • Add elasticsearch-nio jar for base nio classes #27801 (issue: #27802)
  • Add NioGroup for use in different transports #27737 (issue: #27260)
  • Add read timeouts to http module #27713
  • Implement byte array reusage in NioTransport #27696 (issue: #27563)
  • Introduce resizable inbound byte buffer #27551 (issue: #27563)
  • Decouple nio constructs from the tcp transport #27484 (issue: #27260)
  • Remove manual tracking of registered channels #27445 (issue: #27260)
  • Remove tcp profile from low level nio channel #27441 (issue: #27260)
  • Decouple ChannelFactory from Tcp classes #27286 (issue: #27260)
Packaging
  • Introduce Docker images build #36246
  • Move creation of temporary directory to Java #36002 (issue: #31003)
Plugins
  • Plugin install: don’t print download progress in batch mode #36361
Ranking
Recovery
  • Exposed engine must include all operations below global checkpoint during rollback #36159 (issue: #32867)
Rollup
  • Add non-X-Pack centric rollup endpoints #36383 (issues: #35958, #35962)
  • Add more diagnostic stats to job #35471
  • Add wait_for_completion option to StopRollupJob API #34811 (issue: #34574)
Scripting
  • Update joda compat methods to use compat class #36654
  • [Painless] Add boxed type to boxed type casts for method/return #36571
  • [Painless] Add def to boxed type casts #36506
Settings
Search
  • Add copy constructor to SearchRequest #36641 (issue: #32125)
  • Add raw sort values to SearchSortValues transport serialization #36617 (issue: #32125)
  • Add sort and collapse info to SearchHits transport serialization #36555 (issue: #32125)
  • Add default methods to DocValueFormat #36480
  • Respect indices options on _msearch #35887
  • Allow efficient can_match phases on frozen indices #35431 (issues: #34352, #34357)
  • Add a new query type - ScriptScoreQuery #34533 (issues: #23850, #27588, #30303)
Security
Snapshot/Restore
  • Allow Parallel Restore Operations #36397
  • Repo Creation out of ClusterStateTask #36157 (issue: #9488)
  • Add read-only repository verification #35731 (issue: #35703)
SQL
  • Extend the ODBC metric by differentiating between 32 and 64bit platforms #36753 (issue: #36740)
  • Fix wrong appliance of StackOverflow limit for IN #36724 (issue: #36592)
  • Introduce NOW/CURRENT_TIMESTAMP function #36562 (issue: #36534)
  • Move requests' parameters to requests JSON body #36149 (issue: #35992)
  • Make INTERVAL millis optional #36043 (issue: #36032)
  • Implement data type verification for conditionals #35916 (issue: #35907)
  • Implement GREATEST and LEAST functions #35879 (issue: #35878)
  • Implement null safe equality operator <=> #35873 (issue: #35871)
  • SYS COLUMNS returns ODBC specific schema #35870 (issue: #35376)
  • Polish grammar for intervals #35853
  • Add filtering to SYS TYPES #35852 (issue: #35342)
  • Implement NULLIF(expr1, expr2) function #35826 (issue: #35818)
  • Lock down JDBC driver #35798 (issue: #35437)
  • Implement NVL(expr1, expr2) #35794 (issue: #35782)
  • Implement ISNULL(expr1, expr2) #35793 (issue: #35781)
  • Implement IFNULL variant of COALESCE #35762 (issue: #35749)
  • XPack FeatureSet functionality #35725 (issue: #34821)
  • Perform lazy evaluation of mismatched mappings #35676 (issues: #35659, #35675)
  • Improve validation of unsupported fields #35675 (issue: #35673)
  • Move internals from Joda to java.time #35649 (issue: #35633)
Stats
  • Handle OS pretty name on old OS without OS release #35453 (issue: #35440)
Task Management
  • Periodically try to reassign unassigned persistent tasks #36069 (issue: #35792)
  • Only require task permissions #35667 (issue: #35573)
  • Retry if task can’t be written #35054 (issue: #33764)
ZenDiscovery
  • Add discovery types to cluster stats #36442
  • Introduce zen2 discovery type #36298
  • Zen2: Persist cluster states the old way on non-master-eligible nodes #36247 (issue: #3)
  • [Zen2] Storage layer WriteStateException propagation #36052
  • [Zen2] Implement Tombstone REST APIs #36007
  • [Zen2] Update default for USE_ZEN2 to true #35998
  • [Zen2] Add warning if cluster fails to form fast enough #35993
  • [Zen2] Allow Setting a List of Bootstrap Nodes to Wait for #35847
  • [Zen2] VotingTombstone class #35832
  • [Zen2] PersistedState interface implementation #35819
  • [Zen2] Support rolling upgrades from Zen1 #35737
  • [Zen2] Add lag detector #35685
  • [Zen2] Move ClusterState fields to be persisted to ClusterState.Metadata #35625
  • [Zen2] Introduce ClusterBootstrapService #35488
  • [Zen2] Introduce vote withdrawal #35446
  • Zen2: Add basic Zen1 transport-level BWC #35443

Bug fixes

edit
Aggregations
  • fix MultiValuesSourceFieldConfig toXContent #36525 (issue: #36474)
  • Cache the score of the parent document in the nested agg #36019 (issues: #34555, #35985)
  • Correct implemented interface of ParsedReverseNested #35455 (issue: #35449)
  • Handle IndexOrDocValuesQuery in composite aggregation #35392
Audit
  • Fix origin.type for connection_* events #36410
  • Fix IndexAuditTrail rolling restart on rollover edge #35988 (issue: #33867)
Authentication
Build
  • Use explicit deps on test tasks for check #36325
  • Fix jdbc jar pom to not include deps #36036 (issue: #32014)
  • Fix official plugins list #35661 (issue: #35623)
CCR
  • Fix follow stats API’s follower index filtering feature #36647
  • AutoFollowCoordinator should tolerate that auto follow patterns may be removed #35945 (issue: #35937)
  • Only auto follow indices when all primary shards have started #35814 (issue: #35480)
  • Avoid NPE in follower stats when no tasks metadata #35802
  • Fix the names of CCR stats endpoints in usage API #35438
Circuit Breakers
  • Modify BigArrays to take name of circuit breaker #36461 (issue: #31435)
Core
  • Fix CompositeBytesReference#slice to not throw AIOOBE with legal offsets. #35955 (issue: #35950)
  • Suppress CachedTimeThread in hot threads output #35558 (issue: #23175)
  • Upgrade to Joda 2.10.1 #35410 (issue: #33749)
Distributed
  • Combine the execution of an exclusive replica operation with primary term update #36116 (issue: #35850)
  • ActiveShardCount should not fail when closing the index #35936
Engine
  • Set Lucene version upon index creation. #36038 (issue: #33826)
  • Wrap can_match reader with ElasticsearchDirectoryReader #35857
  • Copy checkpoint atomically when rolling generation #35407
Geo
  • More robust handling of ignore_malformed in geoshape parsing #35603 (issues: #34047, #34498)
  • Better handling of malformed geo_points #35554 (issue: #35419)
  • Enables coerce support in WKT polygon parser #35414 (issue: #35059)
Index APIs
  • Fix duplicate phrase in shrink/split error message #36734 (issue: #36729)
  • Raise a 404 exception when document source is not found (#33384) #34083 (issue: #33384)
Ingest
  • Fix on_failure with Drop processor #36686 (issue: #36151)
  • Support default pipelines + bulk upserts #36618 (issue: #36219)
  • Support default pipeline through an alias #36231 (issue: #35817)
License
  • Do not serialize basic license exp in x-pack info #30848
  • Update versions for start_trial after backport #30218 (issue: #30135)
Machine Learning
  • Interrupt Grok in file structure finder timeout #36588
  • Prevent stack overflow while copying ML jobs and datafeeds #36370 (issue: #36360)
  • Adjust file structure finder parser config #35935
  • Fix find_file_structure NPE with should_trim_fields #35465 (issue: #35462)
  • Prevent notifications being created on deletion of a non existent job #35337 (issues: #34058, #35336)
  • Clear Job#finished_time when it is opened (#32605) #32755
  • Fix thread leak when waiting for job flush (#32196) #32541 (issue: #32196)
  • Fix CPoissonMeanConjugate sampling error. #335
Network
  • Do not resolve addresses in remote connection info #36671 (issue: #35658)
  • Always compress based on the settings #36522 (issue: #36399)
  • http.publish_host Should Contain CNAME #32806 (issue: #22029)
  • Adjust SSLDriver behavior for JDK11 changes #32145 (issues: #32122, #32144)
  • Add TRACE, CONNECT, and PATCH http methods #31035 (issue: #31017)
  • Transport client: Don’t validate node in handshake #30737 (issue: #30141)
  • Fix issue with finishing handshake in ssl driver #30580
  • Remove potential nio selector leak #27825
  • Fix issue where the incorrect buffers are written #27695 (issue: #27551)
  • Do not set SO_LINGER on server channels #26997
  • Do not set SO_LINGER to 0 when not shutting down #26871 (issue: #26764)
  • Release pipelined http responses on close #26226
Packaging
  • Fix error message when package install fails due to missing Java #36077 (issue: #31845)
  • Add missing entries to conffiles #35810 (issue: #35691)
Plugins
  • Ensure that azure stream has socket privileges #28751 (issue: #28662)
Recovery
  • Register ResyncTask.Status as a NamedWriteable #36610
Rollup
  • Fix rollup search statistics #36674
Scripting
  • Properly support no-offset date formatting #36316 (issue: #36306)
  • [Painless] Generate Bridge Methods #36097
  • Fix serialization bug in painless execute api request #36075 (issue: #36050)
  • Actually add joda time back to whitelist #35965 (issue: #35915)
  • Add back joda to whitelist #35915 (issue: #35913)
Settings
SQL
  • Fix translation of LIKE/RLIKE keywords #36672 (issues: #36039, #36584)
  • Scripting support for casting functions CAST and CONVERT #36640 (issue: #36061)
  • Fix translation to painless for conditionals #36636 (issue: #36631)
  • Concat should be always not nullable #36601 (issue: #36169)
  • Fix MOD() for long and integer arguments #36599 (issue: #36364)
  • Fix issue with complex HAVING and GROUP BY ordinal #36594 (issue: #36059)
  • Be lenient for tests involving comparison to H2 but strict for csv spec tests #36498 (issue: #36483)
  • Non ISO 8601 versions of DAY_OF_WEEK and WEEK_OF_YEAR functions #36358 (issue: #36263)
  • Do not ignore all fields whose names start with underscore #36214 (issue: #36206)
  • Fix issue with wrong data type for scripted Grouping keys #35969 (issue: #35662)
  • Fix translation of math functions to painless #35910 (issue: #35654)
  • Fix jdbc jar to include deps #35602
  • Fix query translation for scripted queries #35408 (issue: #35232)
  • Clear the cursor if nested inner hits are enough to fulfill the query required limits #35398 (issue: #35176)
  • Introduce IsNull node to simplify expressions #35206 (issues: #34876, #35171)
  • The SSL default configuration shouldn’t override the https protocol if used #34635 (issue: #33817)
  • Minor fix for javadoc #32573 (issue: #32553)
Search
  • Inner hits fail to propagate doc-value format. #36310
  • Fix custom AUTO issue with Fuzziness#toXContent #35807 (issue: #33462)
  • Fix analyzed prefix query in query_string #35756 (issue: #31702)
  • Fix problem with MatchNoDocsQuery in disjunction queries #35726 (issue: #34708)
  • Fix phrase_slop in query_string query #35533 (issue: #35125)
  • Add a More Like This query routing requirement check (#29678) #33974
Security
Snapshot/Restore
  • Upgrade GCS Dependencies to 1.55.0 #36634 (issues: #35229, #35459)
  • Improve Resilience SnapshotShardService #36113 (issue: #32265)
  • Keep SnapshotsInProgress State in Sync with Routing Table #35710
  • Ensure that gcs client creation is privileged #25938 (issue: #25932)
  • Make calls to CloudBlobContainer#exists privileged #25937 (issue: #25931)
Watcher
ZenDiscovery
  • [Zen2] Respect the no_master_block setting #36478
  • Cancel GetDiscoveredNodesAction when bootstrapped #36423 (issues: #36380, #36381)
  • [Zen2] Only elect master-eligible nodes #35996
  • [Zen2] Remove duplicate discovered peers #35505

Regressions

edit
Scripting
  • Use Number as a return value for BucketAggregationScript #35653 (issue: #35351)

Upgrades

edit
Ingest
  • Update geolite2 database in ingest geoip plugin #33840
Network
« Elasticsearch version 7.0.0-beta1 Elasticsearch version 7.0.0-alpha1 »