Script processor

edit

Allows inline and stored scripts to be executed within ingest pipelines.

See How to use scripts to learn more about writing scripts. The Script Processor leverages caching of compiled scripts for improved performance. Since the script specified within the processor is potentially re-compiled per document, it is important to understand how script caching works. To learn more about caching see Script Caching.

Table 31. Script Options

Name Required Default Description

lang

no

"painless"

The scripting language

id

no

-

The stored script id to refer to

source

no

-

An inline script to be executed

params

no

-

Script Parameters

if

no

-

Conditionally execute this processor.

on_failure

no

-

Handle failures for this processor. See Handling Failures in Pipelines.

ignore_failure

no

false

Ignore failures for this processor. See Handling Failures in Pipelines.

tag

no

-

An identifier for this processor. Useful for debugging and metrics.

One of id or source options must be provided in order to properly reference a script to execute.

You can access the current ingest document from within the script context by using the ctx variable.

The following example sets a new field called field_a_plus_b_times_c to be the sum of two existing numeric fields field_a and field_b multiplied by the parameter param_c:

{
  "script": {
    "lang": "painless",
    "source": "ctx.field_a_plus_b_times_c = (ctx.field_a + ctx.field_b) * params.param_c",
    "params": {
      "param_c": 10
    }
  }
}

It is possible to use the Script Processor to manipulate document metadata like _index and _type during ingestion. Here is an example of an Ingest Pipeline that renames the index and type to my-index no matter what was provided in the original index request:

PUT _ingest/pipeline/my-index
{
  "description": "use index:my-index",
  "processors": [
    {
      "script": {
        "source": """
          ctx._index = 'my-index';
          ctx._type = '_doc';
        """
      }
    }
  ]
}

Using the above pipeline, we can attempt to index a document into the any-index index.

PUT any-index/_doc/1?pipeline=my-index
{
  "message": "text"
}

The response from the above index request:

{
  "_index": "my-index",
  "_type": "_doc",
  "_id": "1",
  "_version": 1,
  "result": "created",
  "_shards": {
    "total": 2,
    "successful": 1,
    "failed": 0
  },
  "_seq_no": 89,
  "_primary_term": 1,
}

In the above response, you can see that our document was actually indexed into my-index instead of any-index. This type of manipulation is often convenient in pipelines that have various branches of transformation, and depending on the progress made, indexed into different indices.