Back up a cluster’s configuration

edit

In addition to backing up the data in a cluster, it is important to back up its configuration—​especially when the cluster becomes large and difficult to reconstruct.

Configuration information resides in regular text files on every cluster node. Sensitive setting values such as passwords for the Watcher notification servers, are specified inside a binary secure container, the elasticsearch.keystore file. Some setting values are file paths to the associated configuration data, such as the ingest geo ip database. All these files are contained inside the ES_PATH_CONF directory.

All changes to configuration files are done by manually editing the files or using command line utilities, but not through APIs. In practice, these changes are infrequent after the initial setup.

We recommend that you take regular (ideally, daily) backups of your Elasticsearch config ($ES_PATH_CONF) directory using the file backup software of your choice.

We recommend that you have a configuration management plan for these configuration files. You may wish to check them into version control, or provision them though your choice of configuration management tool.

Some of these files may contain sensitive data such as passwords and TLS keys, therefore you should investigate whether your backup software and/or storage solution are able to encrypt this data.

Some settings in configuration files might be overridden by cluster settings. You can capture these settings in a data backup snapshot by specifying the include_global_state: true (default) parameter for the snapshot API. Alternatively, you can extract these configuration values in text format by using the get settings API:

GET _cluster/settings?pretty&flat_settings&filter_path=persistent

You can store the output of this as a file together with the rest of configuration files.

  • Transient settings are not considered for backup.
  • Elasticsearch security features store configuration data such as role definitions and API keys inside a dedicate special index. This "system" data, complements the security settings configuration and should be backed up as well.
  • Other Elastic Stack components, like Kibana and Machine learning, store their configuration data inside other dedicated indices. From the Elasticsearch perspective these are just data so you can use the regular data backup process.