Set processor

edit

Sets one field and associates it with the specified value. If the field already exists, its value will be replaced with the provided one.

Table 40. Set Options

Name Required Default Description

field

yes

-

The field to insert, upsert, or update. Supports template snippets.

value

yes*

-

The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

copy_from

no

-

The origin field which will be copied to field, cannot set value simultaneously. Supported data types are boolean, number, array, object, string, date, etc.

override

no

true

If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

ignore_empty_value

no

false

If true and used in combination with value which is a template snippet that evaluates to null or an empty string, the processor quietly exits without modifying the document. Similarly, if used in combination with copy_from it will quietly exit if the field does not exist or its value evaluates to null or an empty string.

media_type

no

application/json

The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

description

no

-

Description of the processor. Useful for describing the purpose of the processor or its configuration.

if

no

-

Conditionally execute the processor. See Conditionally run a processor.

ignore_failure

no

false

Ignore failures for the processor. See Handling pipeline failures.

on_failure

no

-

Handle failures for the processor. See Handling pipeline failures.

tag

no

-

Identifier for the processor. Useful for debugging and metrics.

{
  "description" : "sets the value of count to 1",
  "set": {
    "field": "count",
    "value": 1
  }
}

This processor can also be used to copy data from one field to another. For example:

response = client.ingest.put_pipeline(
  id: 'set_os',
  body: {
    description: 'sets the value of host.os.name from the field os',
    processors: [
      {
        set: {
          field: 'host.os.name',
          value: '{{{os}}}'
        }
      }
    ]
  }
)
puts response

response = client.ingest.simulate(
  id: 'set_os',
  body: {
    docs: [
      {
        _source: {
          os: 'Ubuntu'
        }
      }
    ]
  }
)
puts response
PUT _ingest/pipeline/set_os
{
  "description": "sets the value of host.os.name from the field os",
  "processors": [
    {
      "set": {
        "field": "host.os.name",
        "value": "{{{os}}}"
      }
    }
  ]
}

POST _ingest/pipeline/set_os/_simulate
{
  "docs": [
    {
      "_source": {
        "os": "Ubuntu"
      }
    }
  ]
}

Result:

{
  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_id" : "_id",
        "_version" : "-3",
        "_source" : {
          "host" : {
            "os" : {
              "name" : "Ubuntu"
            }
          },
          "os" : "Ubuntu"
        },
        "_ingest" : {
          "timestamp" : "2019-03-11T21:54:37.909224Z"
        }
      }
    }
  ]
}

This processor can also access array fields using dot notation:

response = client.ingest.simulate(
  body: {
    pipeline: {
      processors: [
        {
          set: {
            field: 'my_field',
            value: '{{{input_field.1}}}'
          }
        }
      ]
    },
    docs: [
      {
        _index: 'index',
        _id: 'id',
        _source: {
          input_field: [
            'Ubuntu',
            'Windows',
            'Ventura'
          ]
        }
      }
    ]
  }
)
puts response
POST /_ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [
      {
        "set": {
          "field": "my_field",
          "value": "{{{input_field.1}}}"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "input_field": [
          "Ubuntu",
          "Windows",
          "Ventura"
        ]
      }
    }
  ]
}

Result:

{
  "docs": [
    {
      "doc": {
        "_index": "index",
        "_id": "id",
        "_version": "-3",
        "_source": {
          "input_field": [
            "Ubuntu",
            "Windows",
            "Ventura"
          ],
          "my_field": "Windows"
        },
        "_ingest": {
          "timestamp": "2023-05-05T16:04:16.456475214Z"
        }
      }
    }
  ]
}

The contents of a field including complex values such as arrays and objects can be copied to another field using copy_from:

response = client.ingest.put_pipeline(
  id: 'set_bar',
  body: {
    description: 'sets the value of bar from the field foo',
    processors: [
      {
        set: {
          field: 'bar',
          copy_from: 'foo'
        }
      }
    ]
  }
)
puts response

response = client.ingest.simulate(
  id: 'set_bar',
  body: {
    docs: [
      {
        _source: {
          foo: [
            'foo1',
            'foo2'
          ]
        }
      }
    ]
  }
)
puts response
PUT _ingest/pipeline/set_bar
{
  "description": "sets the value of bar from the field foo",
  "processors": [
    {
      "set": {
        "field": "bar",
        "copy_from": "foo"
      }
    }
  ]
}

POST _ingest/pipeline/set_bar/_simulate
{
  "docs": [
    {
      "_source": {
        "foo": ["foo1", "foo2"]
      }
    }
  ]
}

Result:

{
  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_id" : "_id",
        "_version" : "-3",
        "_source" : {
          "bar": ["foo1", "foo2"],
          "foo": ["foo1", "foo2"]
        },
        "_ingest" : {
          "timestamp" : "2020-09-30T12:55:17.742795Z"
        }
      }
    }
  ]
}