Elasticsearch version 6.1.0

edit

Breaking Changes

edit
Network
  • Allow only a fixed-size receive predictor #26165 (issue: #23185)
REST
Scroll
  • Fail queries with scroll that explicitely set request_cache #27342
Search
  • Add a limit to from + size in top_hits and inner hits. #26492 (issue: #11511)

Breaking Java Changes

edit
Aggregations
  • Moves deferring code into its own subclass #26421
Core
  • Unify Settings xcontent reading and writing #26739
Settings
  • Return List instead of an array from settings #26903
  • Remove Settings,put(Map<String,String>) #26785

Deprecations

edit
Aggregations
  • Deprecate global_ordinals_hash and global_ordinals_low_cardinality #26173 (issue: #26014)
Allocation
  • Add deprecation warning for negative index.unassigned.node_left.delayed_timeout #26832 (issue: #26828)
Analysis
Geo
  • [GEO] 6x Deprecate ShapeBuilders and decouple geojson parse logic #27345
Mapping
  • Deprecate the index_options parameter for numeric fields #26672 (issue: #21475)
Plugin Repository Azure
  • Azure repository: Move to named configurations as we do for S3 repository and secure settings #23405 (issues: #22762, #22763)
Search
  • doc: deprecate _primary and _replica shard option #26792 (issue: #26335)

New Features

edit
Aggregations
  • Aggregations: bucket_sort pipeline aggregation #27152 (issue: #14928)
  • Add composite aggregator #26800
Analysis
  • Added Bengali Analyzer to Elasticsearch with respect to the lucene update #26527
Ingest
Java High Level REST Client
  • Added Delete Index support to high-level REST client #27019 (issue: #25847)
Nested Docs
  • Multi-level Nested Sort with Filters #26395
Query DSL
  • Add terms_set query #27145 (issue: #26915)
  • Introduce sorted_after query for sorted index #26377
  • Add support for auto_generate_synonyms_phrase_query in match_query, multi_match_query, query_string and simple_query_string #26097
Search
Similarities
  • Add a scripted similarity. #25831
Suggesters
  • Expose duplicate removal in the completion suggester #26496 (issue: #23364)
  • Support must and should for context query in context suggester #26407 (issues: #24421, #24565)

Enhancements

edit
Aggregations
  • Allow aggregation sorting via nested aggregation #26683 (issue: #16838)
Allocation
  • Tie-break shard path decision based on total number of shards on path #27039 (issue: #26654)
  • Balance shards for an index more evenly across multiple data paths #26654 (issue: #16763)
  • Expand "NO" decision message in NodeVersionAllocationDecider #26542 (issue: #10403)
  • _reroute’s retry_failed flag should reset failure counter #25888 (issue: #25291)
Analysis
  • Add configurable max_token_length parameter to whitespace tokenizer #26749 (issue: #26643)
CRUD
  • Add wait_for_active_shards parameter to index open command #26682 (issue: #20937)
Core
  • Fix classes that can exit #27518
  • Replace empty index block checks with global block checks in template delete/put actions #27050 (issue: #10530)
  • Allow Uid#decodeId to decode from a byte array slice #26987 (issue: #26931)
  • Use separate searchers for "search visibility" vs "move indexing buffer to disk #26972 (issues: #15768, #26802, #26912, #3593)
  • Add ability to split shards #26931
  • Make circuit breaker mutations debuggable #26067 (issue: #25891)
Dates
Discovery
  • Stop responding to ping requests before master abdication #27329 (issue: #27328)
Engine
  • Ensure external refreshes will also refresh internal searcher to minimize segment creation #27253 (issue: #26972)
  • Move IndexShard#getWritingBytes() under InternalEngine #27209 (issue: #26972)
  • Refactor internal engine #27082
Geo
  • Add ignore_malformed to geo_shape fields #24654 (issue: #23747)
Ingest
  • add json-processor support for non-map json types #27335 (issue: #25972)
  • Introduce templating support to timezone/locale in DateProcessor #27089 (issue: #24024)
  • Add support for parsing inline script (#23824) #26846 (issue: #23824)
  • Consolidate locale parsing. #26400
  • Accept ingest simulate params as ints or strings #23885 (issue: #23823)
Internal
  • Avoid uid creation in ParsedDocument #27241
  • Upgrade to Lucene 7.1.0 snapshot version #26864 (issue: #26527)
  • Remove _index fielddata hack if cluster alias is present #26082 (issue: #25885)
Java High Level REST Client
  • Adjust RestHighLevelClient method modifiers #27238
  • Decouple BulkProcessor from ThreadPool #26727 (issue: #26028)
Logging
  • Add more information on failed_to_convert exception (#21946) #27034 (issue: #21946)
  • Improve shard-failed log messages. #26866
Mapping
  • Allow ip_range to accept CIDR notation #27192 (issue: #26260)
  • Deduplicate _field_names. #26550
  • Throw a better error message for empty field names #26543 (issue: #23348)
  • Stricter validation for min/max values for whole numbers #26137
  • Make FieldMapper.copyTo() always non-null. #25994
Nested Docs
  • Use the primary_term field to identify parent documents #27469 (issue: #24362)
  • Prohibit using nested_filter, nested_path and new nested Option at the same time in FieldSortBuilder #26490 (issue: #17286)
Network
  • Remove manual tracking of registered channels #27445 (issue: #27260)
  • Remove tcp profile from low level nio channel #27441 (issue: #27260)
  • Decouple ChannelFactory from Tcp classes #27286 (issue: #27260)
Percolator
  • Use Lucene’s CoveringQuery to select percolate candidate matches #27271 (issues: #26081, #26307)
  • Add support to percolate query to percolate multiple documents simultaneously #26418
  • Hint what clauses are important in a conjunction query based on fields #26081
  • Add support for selecting percolator query candidate matches containing range queries #25647 (issue: #21040)
Plugin Discovery EC2
  • update AWS SDK for ECS Task IAM support in discovery-ec2 #26479 (issue: #23039)
Plugin Lang Painless
  • Painless: Only allow Painless type names to be the same as the equivalent Java class. #27264
  • Allow for the Painless Definition to have multiple instances for white-listing #27096
  • Separate Painless Whitelist Loading from the Painless Definition #26540
  • Remove Sort enum from Painless Definition #26179
Plugin Repository Azure
  • Add azure storage endpoint suffix #26432 #26568 (issue: #26432)
  • Support for accessing Azure repositories through a proxy #23518 (issues: #23506, #23517)
Plugin Repository S3
Plugins
  • Plugins: Add versionless alias to all security policy codebase properties #26756 (issue: #26521)
  • Allow plugins to plug rescore implementations #26368 (issue: #26208)
Query DSL
Reindex API
  • Update by Query is modified to accept short script parameter. #26841 (issue: #24898)
  • reindex: automatically choose the number of slices #26030 (issues: #24547, #25582)
Rollover
  • Add size-based condition to the index rollover API #27160 (issue: #27004)
  • Add size-based condition to the index rollover API #27115 (issue: #27004)
Scripting
  • Script: Convert script query to a dedicated script context #26003
Search
  • Make fields optional in multi_match query and rely on index.query.default_field by default #27380
  • fix unnecessary logger creation #27349
  • ObjectParser : replace IllegalStateException with ParsingException #27302 (issue: #27147)
  • Uses norms for exists query if enabled #27237
  • Cross-cluster search: make remote clusters optional #27182 (issues: #26118, #27161)
  • Enhances exists queries to reduce need for _field_names #26930 (issue: #26770)
  • Change ParentFieldSubFetchPhase to create doc values iterator once per segment #26815
  • Change VersionFetchSubPhase to create doc values iterator once per segment #26809
  • Change ScriptFieldsFetchSubPhase to create search scripts once per segment #26808 (issue: #26775)
  • Make sure SortBuilders rewrite inner nested sorts #26532
  • Extend testing of build method in ScriptSortBuilder #26520 (issues: #17286, #26490)
  • Accept an array of field names and boosts in the index.query.default_field setting #26320 (issue: #25946)
  • Reject IPv6-mapped IPv4 addresses when using the CIDR notation. #26254 (issue: #26078)
  • Rewrite range queries with open bounds to exists query #26160 (issue: #22640)
Sequence IDs
  • Only fsync global checkpoint if needed #27652
  • Log primary-replica resync failures #27421 (issues: #24841, #27418)
  • Lazy initialize checkpoint tracker bit sets #27179 (issue: #10708)
  • Returns the current primary_term for Get/MultiGet requests #27177 (issue: #26493)
Settings
  • Allow affix settings to specify dependencies #27161
  • Represent lists as actual lists inside Settings #26878 (issue: #26723)
  • Remove Settings#getAsMap() #26845
  • Replace group map settings with affix setting #26819
  • Throw exception if setting isn’t recognized #26569 (issue: #25607)
  • Settings: Move keystore creation to plugin installation #26329 (issue: #26309)
Snapshot/Restore
  • Remove XContentType auto detection in BlobStoreRepository #27480
  • Snapshot: Migrate TransportRequestHandler to TransportMasterNodeAction #27165 (issue: #27151)
  • Fix toString of class SnapshotStatus (#26851) #26852 (issue: #26851)
Stats
  • Adds average document size to DocsStats #27117 (issue: #27004)
  • Stats to record how often the ClusterState diff mechanism is used successfully #27107 (issue: #26973)
  • Expose adaptive replica selection stats in /_nodes/stats API #27090
  • Add cgroup memory usage/limit to OS stats on Linux #26166
  • Add segment attributes to the _segments API. #26157 (issue: #26130)
Suggesters
  • Improve error message for parse failures of completion fields #27297
  • Support AND operation for context query in context suggester #24565 (issue: #24421)

Bug Fixes

edit
Aggregations
  • Disable the "low cardinality" optimization of terms aggregations. #27545 (issue: #27543)
  • scripted_metric _agg parameter disappears if params are provided #27159 (issues: #19768, #19863)
Cluster
  • Properly format IndexGraveyard deletion date as date #27362
  • Remove optimisations to reuse objects when applying a new ClusterState #27317
Core
  • Do not set data paths on no local storage required #27587 (issue: #27572)
  • Ensure threadcontext is preserved when refresh listeners are invoked #27565
  • Ensure logging is configured for CLI commands #27523 (issue: #27521)
  • Protect shard splitting from illegal target shards #27468 (issue: #26931)
  • Avoid NPE when getting build information #27442
  • Fix ShardSplittingQuery to respect nested documents. #27398 (issue: #27378)
  • When building Settings do not set SecureSettings if empty #26988 (issue: #316)
Engine
  • Reset LiveVersionMap on sync commit #27534 (issue: #27516)
  • Carry over version map size to prevent excessive resizing #27516 (issue: #20498)
Geo
  • Correct two equality checks on incomparable types #27688
  • [GEO] fix pointsOnly bug for MULTIPOINT #27415
Index Templates
  • Prevent constructing an index template without index patterns #27662
Ingest
  • Add pipeline support for REST API bulk upsert #27075 (issue: #25601)
  • Fixing Grok pattern for Apache 2.4 #26635
Inner Hits
  • Return an empty _source for nested inner hit when filtering on a field that doesn’t exist #27531
Internal
  • When checking if key exists in ThreadContextStruct:putHeaders() method,should put requestHeaders in map first #26068
  • Adding a refresh listener to a recovering shard should be a noop #26055
Java High Level REST Client
  • Register ip_range aggregation with the high level client #26383
  • add top hits as a parsed aggregation to the rest high level client #26370
Mapping
  • Fix dynamic mapping update generation. #27467
  • Fix merging of _meta field #27352 (issue: #27323)
  • Fixed rounding of bounds in scaled float comparison #27207 (issue: #27189)
Nested Docs
  • Ensure nested documents have consistent version and seq_ids #27455
  • Prevent duplicate fields when mixing parent and root nested includes #27072 (issue: #26990)
Network
  • Throw UOE from compressible bytes stream reset #27564 (issue: #24927)
  • Bubble exceptions when closing compressible streams #27542 (issue: #27540)
  • Do not set SO_LINGER on server channels #26997
  • Do not set SO_LINGER to 0 when not shutting down #26871 (issue: #26764)
  • Close TcpTransport on RST in some Spots to Prevent Leaking TIME_WAIT Sockets #26764 (issue: #26701)
Packaging
  • Removes minimum master nodes default number #26803
  • setgid on /etc/elasticearch on package install #26412 (issue: #26410)
Percolator
  • Avoid TooManyClauses exception if number of terms / ranges is exactly equal to 1024 #27519 (issue: #1)
Plugin Analysis ICU
  • Catch InvalidPathException in IcuCollationTokenFilterFactory #27202
Plugin Lang Painless
  • Painless: Fix variable scoping issue in lambdas #27571 (issue: #26760)
  • Painless: Fix errors allowing void to be assigned to def. #27460 (issue: #27210)
Plugin Repository GCS
  • Create new handlers for every new request in GoogleCloudStorageService #27339 (issue: #27092)
Recovery
  • Flush old indices on primary promotion and relocation #27580 (issue: #27536)
Reindex API
  • Reindex: Fix headers in reindex action #26937 (issue: #22976)
Scroll
  • Fix scroll query with a sort that is a prefix of the index sort #27498
Search
  • Fix profiling naming issues #27133
  • Fix max score tracking with field collapsing #27122 (issue: #23840)
  • Apply missing request options to the expand phase #27118 (issues: #26649, #27079)
  • Calculate and cache result when advanceExact is called #26920 (issue: #26817)
  • Filter unsupported relation for RangeQueryBuilder #26620 (issue: #26575)
  • Handle leniency for phrase query on a field indexed without positions #26388
Sequence IDs
  • Obey translog durability in global checkpoint sync #27641
  • Fix resync request serialization #27418 (issue: #24841)
Settings
  • Allow index settings to be reset by wildcards #27671 (issue: #27537)
Snapshot/Restore
  • Do not swallow exception in ChecksumBlobStoreFormat.writeAtomic() #27597
  • Delete shard store files before restoring a snapshot #27476 (issues: #20220, #26865)
  • Fix snapshot getting stuck in INIT state #27214 (issue: #27180)
  • Fix default value of ignore_unavailable for snapshot REST API (#25359) #27056 (issue: #25359)
  • Do not create directory on readonly repository (#21495) #26909 (issue: #21495)
Stats
  • Include internal refreshes in refresh stats #27615
  • Make Segment statistics aware of segments hold by internal readers #27558
  • Ensure doc_stats are changing even if refresh is disabled #27505

Upgrades

edit
Core
  • Upgrade to Jackson 2.8.10 #27230
  • Upgrade to Lucene 7.1 #27225
Plugin Discovery EC2
Plugin Discovery GCE
  • Update Google SDK to version 1.23.0 #27381 (issue: #26636)
Plugin Lang Painless
  • Upgrade Painless from ANTLR 4.5.1-1 to ANTLR 4.5.3. #27153