IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

« Logstash-to-Logstash Communication Configuring Monitoring for Logstash Nodes »

› ›

Configuring Centralized Pipeline Management

edit

IMPORTANT: This documentation is no longer updated. Refer to Elastic's version policy and the latest documentation.

Configuring Centralized Pipeline Management

edit

To configure centralized pipeline management:

Verify that you are using a license that includes the pipeline management feature.

For more information, see https://www.elastic.co/subscriptions and License Management.
Specify configuration management settings in the logstash.yml file. At a minimum, set:
- xpack.management.enabled: true to enable centralized configuration management.
- xpack.management.elasticsearch.hosts to specify the Elasticsearch instance that will store the Logstash pipeline configurations and metadata.
- xpack.management.pipeline.id to register the pipelines that you want to centrally manage.
Restart Logstash.
If your Elasticsearch cluster is protected with basic authentication, assign the logstash_admin role to any users who will use centralized pipeline management. See X-Pack security.

Centralized management is disabled until you configure and enable X-Pack security.

After you’ve configured Logstash to use centralized pipeline management, you can no longer specify local pipeline configurations. This means that the pipelines.yml file and settings like path.config and config.string are inactive when this feature is enabled.

Configuration Management Settings in Logstash

edit

You can set the following xpack.management settings in logstash.yml to enable centralized pipeline management. For more information about configuring Logstash, see logstash.yml.

The following example shows basic settings that assume Elasticsearch and Kibana are installed on the localhost with basic AUTH enabled, but no SSL. If you’re using SSL, you need to specify additional SSL settings.

xpack.management.enabled: true
xpack.management.elasticsearch.hosts: "http://localhost:9200/"
xpack.management.elasticsearch.username: logstash_admin_user
xpack.management.elasticsearch.password: t0p.s3cr3t
xpack.management.logstash.poll_interval: 5s
xpack.management.pipeline.id: ["apache", "cloudwatch_logs"]

xpack.management.enabled: Set to true to enable X-Pack centralized configuration management for Logstash.
xpack.management.logstash.poll_interval: How often the Logstash instance polls for pipeline changes from Elasticsearch. The default is 5s.
xpack.management.pipeline.id: Specify a comma-separated list of pipeline IDs to register for centralized pipeline management. After changing this setting, you need to restart Logstash to pick up changes.
xpack.management.elasticsearch.hosts: The Elasticsearch instance that will store the Logstash pipeline configurations and metadata. This might be the same Elasticsearch instance specified in the outputs section in your Logstash configuration, or a different one. Defaults to http://localhost:9200.
xpack.management.elasticsearch.username and xpack.management.elasticsearch.password: If your Elasticsearch cluster is protected with basic authentication, these settings provide the username and password that the Logstash instance uses to authenticate for accessing the configuration data. The username you specify here should have the logstash_admin role, which provides access to .logstash-* indices for managing configurations.
xpack.management.elasticsearch.ssl.certificate_authority: Optional setting that enables you to specify a path to the .pem file for the certificate authority for your Elasticsearch instance.
xpack.management.elasticsearch.ssl.truststore.path: Optional setting that provides the path to the Java keystore (JKS) to validate the server’s certificate.
xpack.management.elasticsearch.ssl.truststore.password: Optional setting that provides the password to the truststore.
xpack.management.elasticsearch.ssl.keystore.path: Optional setting that provides the path to the Java keystore (JKS) to validate the client’s certificate.
xpack.management.elasticsearch.ssl.keystore.password: Optional setting that provides the password to the keystore.

« Logstash-to-Logstash Communication Configuring Monitoring for Logstash Nodes »