Secrets keystore for secure settings

edit

When you configure Logstash, you might need to specify sensitive settings or configuration, such as passwords. Rather than relying on file system permissions to protect these values, you can use the Logstash keystore to securely store secret values for use in configuration settings.

After adding a key and its secret value to the keystore, you can use the key in place of the secret value when you configure sensitive settings.

The syntax for referencing keys is identical to the syntax for environment variables:

${KEY}

Where KEY is the name of the key.

For example, imagine that the keystore contains a key called ES_PWD with the value yourelasticsearchpassword:

  • In configuration files, use: output { elasticsearch {...password => "${ES_PWD}" } } }
  • In logstash.yml, use: xpack.management.elasticsearch.password: ${ES_PWD}

Notice that the Logstash keystore differs from the Elasticsearch keystore. Whereas the Elasticsearch keystore lets you store elasticsearch.yml values by name, the Logstash keystore lets you specify arbitrary names that you can reference in the Logstash configuration.

There are some configuration fields that have no secret meaning, so not every field could leverage the secret store for variables substitution. Plugin’s id field is a field of this kind

Referencing keystore data from pipelines.yml or the command line (-e) is not currently supported.

Referencing keystore data from centralized pipeline management requires each Logstash deployment to have a local copy of the keystore.

When Logstash parses the settings (logstash.yml) or configuration (/etc/logstash/conf.d/*.conf), it resolves keys from the keystore before resolving environment variables.

Keystore password

edit

You can protect access to the Logstash keystore by storing a password in an environment variable called LOGSTASH_KEYSTORE_PASS. If you create the Logstash keystore after setting this variable, the keystore will be password protected. This means that the environment variable needs to be accessible to the running instance of Logstash. This environment variable must also be correctly set for any users who need to issue keystore commands (add, list, remove, etc.).

Using a keystore password is recommended, but optional. The data will be encrypted even if you do not set a password. However, it is highly recommended to configure the keystore password and grant restrictive permissions to any files that may contain the environment variable value. If you choose not to set a password, then you can skip the rest of this section.

For example:

set +o history
export LOGSTASH_KEYSTORE_PASS=mypassword
set -o history
bin/logstash-keystore create

This setup requires the user running Logstash to have the environment variable LOGSTASH_KEYSTORE_PASS=mypassword defined. If the environment variable is not defined, Logstash cannot access the the keystore.

When you run Logstash from an RPM or DEB package installation, the environment variables are sourced from /etc/sysconfig/logstash.

You might need to create /etc/sysconfig/logstash. This file should be owned by root with 600 permissions. The expected format of /etc/sysconfig/logstash is ENVIRONMENT_VARIABLE=VALUE, with one entry per line.

For other distributions, such as Docker or ZIP, see the documentation for your runtime environment (Windows, Docker, etc) to learn how to set the environment variable for the user that runs Logstash. Ensure that the environment variable (and thus the password) is only accessible to that user.

Keystore location

edit

The keystore must be located in Logstash’s path.settings directory. This is the same directory that contains the logstash.yml file. When performing any operation against the keystore, it is recommended to set path.settings for the keystore command. For example, to create a keystore on a RPM/DEB installation:

set +o history
export LOGSTASH_KEYSTORE_PASS=mypassword
set -o history
sudo -E /usr/share/logstash/bin/logstash-keystore --path.settings /etc/logstash create

See Logstash Directory Layout for more about the default directory locations.

You will see a warning if the path.settings is not pointed to the same directory as the logstash.yml.

Create a keystore

edit

To create a secrets keystore, use the create:

bin/logstash-keystore create

Creates the keystore in the directory defined by the path.settings configuration setting.

It is recommended that you set a keystore password when creating the keystore.

Add keys

edit

To store sensitive values, such as authentication credentials for Elasticsearch, use the add command:

bin/logstash-keystore add ES_PWD

When prompted, enter a value for the key.

List keys

edit

To list the keys defined in the keystore, use:

bin/logstash-keystore list

Remove keys

edit

To remove a key from the keystore, use:

bin/logstash-keystore remove ES_PWD