- Elasticsearch Guide: other versions:
- Getting Started
- Setup Elasticsearch
- Breaking changes
- Breaking changes in 5.0
- Search and Query DSL changes
- Mapping changes
- Percolator changes
- Suggester changes
- Index APIs changes
- Document API changes
- Settings changes
- Allocation changes
- HTTP changes
- REST API changes
- CAT API changes
- Java API changes
- Packaging
- Plugin changes
- Filesystem related changes
- Path to data on disk
- Aggregation changes
- Script related changes
- Breaking changes in 5.0
- API Conventions
- Document APIs
- Search APIs
- Aggregations
- Metrics Aggregations
- Avg Aggregation
- Cardinality Aggregation
- Extended Stats Aggregation
- Geo Bounds Aggregation
- Geo Centroid Aggregation
- Max Aggregation
- Min Aggregation
- Percentiles Aggregation
- Percentile Ranks Aggregation
- Scripted Metric Aggregation
- Stats Aggregation
- Sum Aggregation
- Top hits Aggregation
- Value Count Aggregation
- Bucket Aggregations
- Children Aggregation
- Date Histogram Aggregation
- Date Range Aggregation
- Diversified Sampler Aggregation
- Filter Aggregation
- Filters Aggregation
- Geo Distance Aggregation
- GeoHash grid Aggregation
- Global Aggregation
- Histogram Aggregation
- IP Range Aggregation
- Missing Aggregation
- Nested Aggregation
- Range Aggregation
- Reverse nested Aggregation
- Sampler Aggregation
- Significant Terms Aggregation
- Terms Aggregation
- Pipeline Aggregations
- Avg Bucket Aggregation
- Derivative Aggregation
- Max Bucket Aggregation
- Min Bucket Aggregation
- Sum Bucket Aggregation
- Stats Bucket Aggregation
- Extended Stats Bucket Aggregation
- Percentiles Bucket Aggregation
- Moving Average Aggregation
- Cumulative Sum Aggregation
- Bucket Script Aggregation
- Bucket Selector Aggregation
- Serial Differencing Aggregation
- Matrix Aggregations
- Caching heavy aggregations
- Returning only aggregation results
- Aggregation Metadata
- Metrics Aggregations
- Indices APIs
- Create Index
- Delete Index
- Get Index
- Indices Exists
- Open / Close Index API
- Shrink Index
- Rollover Index
- Put Mapping
- Get Mapping
- Get Field Mapping
- Types Exists
- Index Aliases
- Update Indices Settings
- Get Settings
- Analyze
- Index Templates
- Shadow replica indices
- Indices Stats
- Indices Segments
- Indices Recovery
- Indices Shard Stores
- Clear Cache
- Flush
- Refresh
- Force Merge
- cat APIs
- Cluster APIs
- Query DSL
- Mapping
- Analysis
- Anatomy of an analyzer
- Testing analyzers
- Analyzers
- Tokenizers
- Token Filters
- Standard Token Filter
- ASCII Folding Token Filter
- Length Token Filter
- Lowercase Token Filter
- Uppercase Token Filter
- NGram Token Filter
- Edge NGram Token Filter
- Porter Stem Token Filter
- Shingle Token Filter
- Stop Token Filter
- Word Delimiter Token Filter
- Stemmer Token Filter
- Stemmer Override Token Filter
- Keyword Marker Token Filter
- Keyword Repeat Token Filter
- KStem Token Filter
- Snowball Token Filter
- Phonetic Token Filter
- Synonym Token Filter
- Compound Word Token Filter
- Reverse Token Filter
- Elision Token Filter
- Truncate Token Filter
- Unique Token Filter
- Pattern Capture Token Filter
- Pattern Replace Token Filter
- Trim Token Filter
- Limit Token Count Token Filter
- Hunspell Token Filter
- Common Grams Token Filter
- Normalization Token Filter
- CJK Width Token Filter
- CJK Bigram Token Filter
- Delimited Payload Token Filter
- Keep Words Token Filter
- Keep Types Token Filter
- Classic Token Filter
- Apostrophe Token Filter
- Decimal Digit Token Filter
- Fingerprint Token Filter
- Minhash Token Filter
- Character Filters
- Modules
- Index Modules
- Ingest Node
- Pipeline Definition
- Ingest APIs
- Accessing Data in Pipelines
- Handling Failures in Pipelines
- Processors
- Append Processor
- Convert Processor
- Date Processor
- Date Index Name Processor
- Fail Processor
- Foreach Processor
- Grok Processor
- Gsub Processor
- Join Processor
- JSON Processor
- Lowercase Processor
- Remove Processor
- Rename Processor
- Script Processor
- Set Processor
- Split Processor
- Sort Processor
- Trim Processor
- Uppercase Processor
- Dot Expander Processor
- How To
- Testing
- Glossary of terms
- Release Notes
- 5.0.2 Release Notes
- 5.0.1 Release Notes
- 5.0.0 Combined Release Notes
- 5.0.0 GA Release Notes
- 5.0.0-rc1 Release Notes
- 5.0.0-beta1 Release Notes
- 5.0.0-alpha5 Release Notes
- 5.0.0-alpha4 Release Notes
- 5.0.0-alpha3 Release Notes
- 5.0.0-alpha2 Release Notes
- 5.0.0-alpha1 Release Notes
- 5.0.0-alpha1 Release Notes (Changes previously released in 2.x)
WARNING: Version 5.0 of Elasticsearch has passed its EOL date.
This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.
Script related changes
editScript related changes
editSwitched Default Language from Groovy to Painless
editThe default scripting language for Elasticsearch is now Painless. Painless is a custom-built language with syntax similar to Groovy designed to be fast as well as secure. Many Groovy scripts will be identitical to Painless scripts to help make the transition between languages as simple as possible.
Documentation for Painless can be found at Painless Scripting Language
One common difference to note between Groovy and Painless is the use of parameters — all parameters in Painless
must be prefixed with params.
now. The following example shows the difference:
Groovy:
{ "script_score": { "script": { "lang": "groovy", "inline": "Math.log(_score * 2) + my_modifier", "params": { "my_modifier": 8 } } } }
Painless (my_modifer
is prefixed with params
):
{ "script_score": { "script": { "lang": "painless", "inline": "Math.log(_score * 2) + params.my_modifier", "params": { "my_modifier": 8 } } } }
The script.default_lang
setting has been removed. It is no longer possible set the default scripting language. If a
different language than painless
is used then this should be explicitly specified on the script itself.
For scripts with no explicit language defined, that are part of already stored percolator queries, the default language
can be controlled with the script.legacy.default_lang
setting.
Removed 1.x script and template syntax
editThe deprecated 1.x syntax of defining inline scripts / templates and referring to file or index base scripts / templates have been removed.
The script
and params
string parameters can no longer be used and instead the script
object syntax must be used.
This applies for the update api, script sort, script_score
function, script
query, scripted_metric
aggregation and
script_heuristic
aggregation.
So this usage of inline scripts is no longer allowed:
{ "script_score": { "lang": "groovy", "script": "Math.log(_score * 2) + my_modifier", "params": { "my_modifier": 8 } } }
and instead this syntax must be used:
{ "script_score": { "script": { "lang": "groovy", "inline": "Math.log(_score * 2) + my_modifier", "params": { "my_modifier": 8 } } } }
The script
or script_file
parameter can no longer be used to refer to file based scripts and templates and instead
file
must be used.
This usage of referring to file based scripts is no longer valid:
{ "script_score": { "script": "calculate-score", "params": { "my_modifier": 8 } } }
This usage is valid:
{ "script_score": { "script": { "lang": "groovy", "file": "calculate-score", "params": { "my_modifier": 8 } } } }
The script_id
parameter can no longer be used the refer to indexed based scripts and templates and instead id
must
be used.
This usage of referring to indexed scripts is no longer valid:
{ "script_score": { "script_id": "indexedCalculateScore", "params": { "my_modifier": 8 } } }
This usage is valid:
{ "script_score": { "script": { "id": "indexedCalculateScore", "lang" : "groovy", "params": { "my_modifier": 8 } } } }
Template query
editThe query
field in the template
query can no longer be used.
This 1.x syntax can no longer be used:
{ "query": { "template": { "query": {"match_{{template}}": {}}, "params" : { "template" : "all" } } } }
and instead the following syntax should be used:
{ "query": { "template": { "inline": {"match_{{template}}": {}}, "params" : { "template" : "all" } } } }
Search templates
editThe top level template
field in the search template api has been replaced with consistent template / script object
syntax. This 1.x syntax can no longer be used:
{ "template" : { "query": { "match" : { "{{my_field}}" : "{{my_value}}" } }, "size" : "{{my_size}}" }, "params" : { "my_field" : "foo", "my_value" : "bar", "my_size" : 5 } }
and instead the following syntax should be used:
{ "inline" : { "query": { "match" : { "{{my_field}}" : "{{my_value}}" } }, "size" : "{{my_size}}" }, "params" : { "my_field" : "foo", "my_value" : "bar", "my_size" : 5 } }
Indexed scripts and templates
editIndexed scripts and templates have been replaced by stored scripts
which stores the scripts and templates in the cluster state instead of a dedicate .scripts
index.
For the size of stored scripts there is a soft limit of 65535 bytes. If scripts exceed that size then
the script.max_size_in_bytes
setting can be added to elasticsearch.yml to change the soft limit to a higher value.
If scripts are really large, other options like native scripts should be considered.
Previously indexed scripts in the .scripts
index will not be used any more as
Elasticsearch will now try to fetch the scripts from the cluster state. Upon upgrading
to 5.x the .scripts
index will remain to exist, so it can be used by a script to migrate
the stored scripts from the .scripts
index into the cluster state. The current format of the scripts
and templates hasn’t been changed, only the 1.x format has been removed.
Python migration script
editThe following Python script can be used to import your indexed scripts into the cluster state as stored scripts:
from elasticsearch import Elasticsearch,helpers es = Elasticsearch([ {'host': 'localhost'} ]) for doc in helpers.scan(es, index=".scripts", preserve_order=True): es.put_script(lang=doc['_type'], id=doc['_id'], body=doc['_source'])
This script makes use of the official Elasticsearch Python client and
therefore you need to make sure that your have installed the client in your
environment. For more information on this please see
elasticsearch-py
.
Perl migration script
editThe following Perl script can be used to import your indexed scripts into the cluster state as stored scripts:
use Search::Elasticsearch; my $es = Search::Elasticsearch->new( nodes => 'localhost:9200'); my $scroll = $es->scroll_helper( index => '.scripts', sort => '_doc'); while (my $doc = $scroll->next) { $e->put_script( lang => $doc->{_type}, id => $doc->{_id}, body => $doc->{_source} ); }
This script makes use of the official Elasticsearch Perl client and
therefore you need to make sure that your have installed the client in your
environment. For more information on this please see
Search::Elasticsearch
.
Verifying script migration
editAfter you have moved the scripts via the provided script or otherwise then you can verify with the following request if the migration has happened successfully:
GET _cluster/state?filter_path=metadata.stored_scripts
The response should include all your scripts from the .scripts
index.
After you have verified that all your scripts have been moved, optionally as a last step,
you can delete the .scripts
index as Elasticsearch no longer uses it.
Indexed scripts Java APIs
editAll the methods related to interacting with indexed scripts have been removed.
The Java API methods for interacting with stored scripts have been added under ClusterAdminClient
class.
The sugar methods that used to exist on the indexed scripts API methods don’t exist on the methods for
stored scripts. The only way to provide scripts is by using BytesReference
implementation, if a string needs to be
provided the BytesArray
class should be used.
Scripting engines now register only a single language
editPrior to 5.0.0, script engines could register multiple languages. The Javascript
script engine in particular registered both "lang": "js"
and "lang":
"javascript"
. Script engines can now only register a single language. All
references to "lang": "js"
should be changed to "lang": "javascript"
for
existing users of the lang-javascript plugin.
Scripting engines now register only a single extension
editPrior to 5.0.0 scripting engines could register multiple extensions. The only engine doing this was the Javascript engine, which registered "js" and "javascript". It now only registers the "js" file extension for on-disk scripts.
.javascript
files are no longer supported (use .js
)
editThe Javascript engine previously registered "js" and "javascript". It now only registers the "js" file extension for on-disk scripts.
Removed scripting query string parameters from update rest api
editThe script
, script_id
and scripting_upsert
query string parameters have been removed from the update api.
Java transport client
editThe TemplateQueryBuilder
has been moved to the lang-mustache
module.
Therefor when using the TemplateQueryBuilder
from the Java native client the
lang-mustache module should be on the classpath. Also the transport client
should load the lang-mustache module as plugin:
TransportClient transportClient = TransportClient.builder() .settings(Settings.builder().put("node.name", "node")) .addPlugin(MustachePlugin.class) .build(); transportClient.addTransportAddress( new InetSocketTransportAddress(new InetSocketAddress(InetAddresses.forString("127.0.0.1"), 9300)) );
Also the helper methods in QueryBuilders
class that create a TemplateQueryBuilder
instance have been removed,
instead the constructors on TemplateQueryBuilder
should be used.
Template query
editThe template
query has been deprecated in favour of the search template api. The template
query is scheduled
to be removed in the next major version.
GeoPoint scripts
editThe following helper methods have been removed from GeoPoint scripting:
-
factorDistance
-
factorDistanceWithDefault
-
factorDistance02
-
factorDistance13
-
arcDistanceInKm
-
arcDistanceInKmWithDefault
-
arcDistanceInMiles
-
arcDistanceInMilesWithDefault
-
distanceWithDefault
-
distanceInKm
-
distanceInKmWithDefault
-
distanceInMiles
-
distanceInMilesWithDefault
-
geohashDistanceInKm
-
geohashDistanceInMiles
Instead use arcDistance
, arcDistanceWithDefault
, planeDistance
, planeDistanceWithDefault
, geohashDistance
,
geohashDistanceWithDefault
and convert from default units (meters) to desired units using the appropriate constance
(e.g., multiply by 0.001
to convert to Km).
On this page
- Switched Default Language from Groovy to Painless
- Removed 1.x script and template syntax
- Template query
- Search templates
- Indexed scripts and templates
- Python migration script
- Perl migration script
- Verifying script migration
- Indexed scripts Java APIs
- Scripting engines now register only a single language
- Scripting engines now register only a single extension
.javascript
files are no longer supported (use.js
)- Removed scripting query string parameters from update rest api
- Java transport client
- Template query
- GeoPoint scripts