Elasticsearch_java output plugin v2.1.4

The Elasticsearch action to perform. Valid actions are:

For more details on actions, check out the Elasticsearch bulk API documentation

The name of your cluster if you set it on the Elasticsearch side. Useful for discovery when using node or transport protocols. By default, it looks for a cluster named elasticsearch. Equivalent to the Elasticsearch option cluster.name

Enable doc_as_upsert for update mode. Create a new document with source if document_id doesn’t exist in Elasticsearch

The document ID for the index. Useful for overwriting existing entries in Elasticsearch with the same ID.

The document type to write events to. Generally you should try to write only similar events to the same type. String expansion %{foo} works here. Unless you set document_type, the event type will be used if it exists otherwise the document type will be assigned the value of logs

Sets the host(s) of the remote instance. If given an array it will load balance requests across the hosts specified in the hosts parameter. Remember the http protocol uses the http address (eg. 9200, not 9300). "127.0.0.1" ["127.0.0.1:9200","127.0.0.2:9200"] ["http://127.0.0.1"] ["https://127.0.0.1:9200"] ["https://127.0.0.1:9200/mypath"] (If using a proxy on a subpath) It is important to exclude dedicated master nodes from the hosts list to prevent LS from sending bulk requests to the master nodes. So this parameter should only reference either data or client nodes in Elasticsearch.

Any special characters present in the URLs here MUST be URL escaped! This means # should be put in as %23 for instance.

The index to write events to. This can be dynamic using the %{foo} syntax. The default value will partition your indices by day so you can more easily delete old data or only search specific date ranges. Indexes may not contain uppercase characters. For weekly indexes ISO 8601 format is recommended, eg. logstash-%{+xxxx.ww}. LS uses Joda to format the index pattern from event timestamp. Joda formats are defined here.

From Logstash 1.3 onwards, a template is applied to Elasticsearch during Logstash’s startup if one with the name template_name does not already exist. By default, the contents of this template is the default template for logstash-%{+YYYY.MM.dd} which always matches indices based on the pattern logstash-*. Should you require support for other index names, or would like to change the mappings in the template in general, a custom template can be specified by setting template to the path of a template file.

Setting manage_template to false disables this feature. If you require more control over template creation, (e.g. creating indices dynamically based on field names) you should set manage_template to false and use the REST API to apply your templates manually.

This setting no longer does anything. It exists to keep config validation from failing. It will be removed in future versions.

The name/address of the host to bind to for Elasticsearch clustering. Equivalent to the Elasticsearch option network.host option. This MUST be set for either protocol to work (node or transport)! The internal Elasticsearch node will bind to this ip. This ip MUST be reachable by all nodes in the Elasticsearch cluster

The node name Elasticsearch will use when joining a cluster.

By default, this is generated internally by the ES client.

For child documents, ID of the associated parent. This can be dynamic using the %{foo} syntax.

Set which ingest pipeline you wish to execute for an event. You can also use event dependent configuration here like pipeline => "%{INGEST_PIPELINE}"

Choose the protocol used to talk to Elasticsearch.

The node protocol (default) will connect to the cluster as a normal Elasticsearch node (but will not store data). If you use the node protocol, you must permit bidirectional communication on the port 9300 (or whichever port you have configured).

If you do not specify the host parameter, it will use multicast for Elasticsearch discovery. While this may work in a test/dev environment where multicast is enabled in Elasticsearch, we strongly recommend using unicast in Elasticsearch. To connect to an Elasticsearch cluster with unicast, you must include the host parameter (see relevant section above).

The transport protocol will connect to the host you specify and will not show up as a node in the Elasticsearch cluster. This is useful in situations where you cannot permit connections outbound from the Elasticsearch cluster to this Logstash server.

All protocols will use bulk requests when talking to Elasticsearch.

Set initial interval in seconds between bulk retries. Doubled on each retry up to retry_max_interval

Set max interval in seconds between bulk retries.

The number of times Elasticsearch should internally retry an update/upserted document See the partial updates for more info

A routing override to be applied to all processed events. This can be dynamic using the %{foo} syntax.

Set script name for scripted update mode

Set the language of the used script. If not set, this defaults to painless in ES 5.0

Define the type of script referenced by "script" variable inline : "script" contains inline script indexed : "script" contains the name of script directly indexed in elasticsearch file : "script" contains the name of script stored in elasticseach’s config directory

Set variable name passed to script (scripted update)

if enabled, script is in charge of creating non-existent document (scripted update)

Enable cluster sniffing (transport only). Asks host for the list of all cluster nodes and adds them to the hosts list Equivalent to the Elasticsearch option client.transport.sniff

You can set the path to your own template here, if you so desire. If not set, the included template will be used.

This configuration option defines how the template is named inside Elasticsearch. Note that if you have used the template management features and subsequently change this, you will need to prune the old template manually, e.g.

curl -XDELETE <http://localhost:9200/_template/OldTemplateName?pretty>

where OldTemplateName is whatever the former setting was.

The template_overwrite option will always overwrite the indicated template in Elasticsearch with either the one indicated by template or the included one. This option is set to false by default. If you always want to stay up to date with the template provided by Logstash, this option could be very useful to you. Likewise, if you have your own template file managed by puppet, for example, and you wanted to be able to update it regularly, this option could help there as well.

Please note that if you are using your own customized version of the Logstash template (logstash), setting this to true will make Logstash to overwrite the "logstash" template (i.e. removing all customized settings)

This sets the local port to bind to. Equivalent to the Elasticsrearch option transport.tcp.port

Set upsert content for update mode.s Create a new document with this parameter as json string if document_id doesn’t exists

The version to use for indexing. Use sprintf syntax like %{my_version} to use a field value here. See https://www.elastic.co/blog/elasticsearch-versioning-support.

The version_type to use for indexing. See https://www.elastic.co/blog/elasticsearch-versioning-support. See also https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#_version_types

The codec used for output data. Output codecs are a convenient method for encoding your data before it leaves the output without needing a separate filter in your Logstash pipeline.

Disable or enable metric logging for this specific plugin instance. By default we record all the metrics we can, but you can disable metrics collection for a specific plugin.

Add a unique ID to the plugin configuration. If no ID is specified, Logstash will generate one. It is strongly recommended to set this ID in your configuration. This is particularly useful when you have two or more plugins of the same type. For example, if you have 2 elasticsearch_java outputs. Adding a named ID in this case will help in monitoring Logstash when using the monitoring APIs.

output {
  elasticsearch_java {
    id => "my_plugin_id"
  }
}