This documentation contains work-in-progress information for future Elastic Stack and Cloud releases. Use the version selector to view supported release docs. It also contains some Elastic Cloud serverless information. Check out our serverless docs for more details.
« Ingest APIs Get pipeline API »
Elastic Docs ›Elasticsearch Guide [master] ›REST APIs ›Ingest APIs

Create or update pipeline API

edit

Create or update pipeline API

edit

Creates or updates an ingest pipeline. Changes made using this API take effect immediately.

resp = client.ingest.put_pipeline(
    id="my-pipeline-id",
    description="My optional pipeline description",
    processors=[
        {
            "set": {
                "description": "My optional processor description",
                "field": "my-keyword-field",
                "value": "foo"
            }
        }
    ],
)
print(resp)
response = client.ingest.put_pipeline(
  id: 'my-pipeline-id',
  body: {
    description: 'My optional pipeline description',
    processors: [
      {
        set: {
          description: 'My optional processor description',
          field: 'my-keyword-field',
          value: 'foo'
        }
      }
    ]
  }
)
puts response
const response = await client.ingest.putPipeline({
  id: "my-pipeline-id",
  description: "My optional pipeline description",
  processors: [
    {
      set: {
        description: "My optional processor description",
        field: "my-keyword-field",
        value: "foo",
      },
    },
  ],
});
console.log(response);
PUT _ingest/pipeline/my-pipeline-id
{
  "description" : "My optional pipeline description",
  "processors" : [
    {
      "set" : {
        "description" : "My optional processor description",
        "field": "my-keyword-field",
        "value": "foo"
      }
    }
  ]
}

Request

edit

PUT /_ingest/pipeline/<pipeline>

Prerequisites

edit
  • If the Elasticsearch security features are enabled, you must have the manage_pipeline, manage_ingest_pipelines, or manage cluster privilege to use this API.

Path parameters

edit
<pipeline>

(Required, string) ID of the ingest pipeline to create or update.

To avoid naming collisions with built-in and Fleet-managed ingest pipelines, avoid using @ as part of your own ingest pipelines names. The exception of that rule are the *@custom ingest pipelines that let you safely add a custom pipeline to managed pipelines. See also Pipelines for Fleet and Elastic Agent.

Query parameters

edit
if_version
(Optional, integer) Perform the operation only if the pipeline has this version. If specified and the update is successful, the pipeline’s version is incremented.
master_timeout
(Optional, time units) Period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. Defaults to 30s. Can also be set to -1 to indicate that the request should never timeout.
timeout
(Optional, time units) Period to wait for a response from all relevant nodes in the cluster after updating the cluster metadata. If no response is received before the timeout expires, the cluster metadata update still applies but the response will indicate that it was not completely acknowledged. Defaults to 30s. Can also be set to -1 to indicate that the request should never timeout.

Request body

edit
description
(Optional, string) Description of the ingest pipeline.
on_failure

(Optional, array of processor objects) Processors to run immediately after a processor failure.

Each processor supports a processor-level on_failure value. If a processor without an on_failure value fails, Elasticsearch uses this pipeline-level parameter as a fallback. The processors in this parameter run sequentially in the order specified. Elasticsearch will not attempt to run the pipeline’s remaining processors.

processors
(Required, array of processor objects) Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.
version

(Optional, integer) Version number used by external systems to track ingest pipelines.

See the if_version parameter above for how the version attribute is used.

_meta
(Optional, object) Optional metadata about the ingest pipeline. May have any contents. This map is not automatically generated by Elasticsearch.
deprecated
(Optional, boolean) Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

Examples

edit

Pipeline metadata

edit

You can use the _meta parameter to add arbitrary metadata to a pipeline. This user-defined object is stored in the cluster state, so keeping it short is preferable.

The _meta parameter is optional and not automatically generated or used by Elasticsearch.

To unset _meta, replace the pipeline without specifying one.

resp = client.ingest.put_pipeline(
    id="my-pipeline-id",
    description="My optional pipeline description",
    processors=[
        {
            "set": {
                "description": "My optional processor description",
                "field": "my-keyword-field",
                "value": "foo"
            }
        }
    ],
    meta={
        "reason": "set my-keyword-field to foo",
        "serialization": {
            "class": "MyPipeline",
            "id": 10
        }
    },
)
print(resp)
response = client.ingest.put_pipeline(
  id: 'my-pipeline-id',
  body: {
    description: 'My optional pipeline description',
    processors: [
      {
        set: {
          description: 'My optional processor description',
          field: 'my-keyword-field',
          value: 'foo'
        }
      }
    ],
    _meta: {
      reason: 'set my-keyword-field to foo',
      serialization: {
        class: 'MyPipeline',
        id: 10
      }
    }
  }
)
puts response
const response = await client.ingest.putPipeline({
  id: "my-pipeline-id",
  description: "My optional pipeline description",
  processors: [
    {
      set: {
        description: "My optional processor description",
        field: "my-keyword-field",
        value: "foo",
      },
    },
  ],
  _meta: {
    reason: "set my-keyword-field to foo",
    serialization: {
      class: "MyPipeline",
      id: 10,
    },
  },
});
console.log(response);
PUT /_ingest/pipeline/my-pipeline-id
{
  "description" : "My optional pipeline description",
  "processors" : [
    {
      "set" : {
        "description" : "My optional processor description",
        "field": "my-keyword-field",
        "value": "foo"
      }
    }
  ],
  "_meta": {
    "reason": "set my-keyword-field to foo",
    "serialization": {
      "class": "MyPipeline",
      "id": 10
    }
  }
}

To check the _meta, use the get pipeline API.

« Ingest APIs Get pipeline API »