Streamline configuration processes with an official Elastic Stack Terraform provider
We’re excited to announce that operation and SRE teams can now take advantage of an official Elastic Stack Terraform provider to configure the Elastic Stack across all types of infrastructure, whether on Elastic Cloud, or on-premises and self-managed.
This Elastic Stack Terraform provider will allow you to fully control the configuration of Elasticsearch, Kibana, Fleet, and any other component within the Elastic Stack.
The Terraform provider helps you safely manage production-related infrastructure using methodologies like infrastructure as code, which allow you to apply peer-reviewed infrastructure changes in an automated and controlled fashion.
Let’s walk through an example of how to spin up an Elastic Cloud deployment, then immediately configure it with an index lifecycle policy — using the new Elastic Stack Terraform Provider in a single Terraform script.
Getting started
To get started, include the following code block in one of your Terraform .tf configuration files:terraform {
required_providers {
elasticstack = {
source = "elastic/elasticstack"
version = "0.2.0"
}
}
}
The provider will be downloaded from the Terraform registry once you initialize it using terraform init
.
Connecting to an Elastic Stack can be configured either in the provider block like so:
provider "elasticstack" {
elasticsearch {
username = "elastic"
password = "changeme"
endpoints = ["http://localhost:9200"]
}
}
Or, by using an elasticsearch_connection
block with the same parameters on a per-resource level. This lets you use the provider to configure multiple Elastic Stack instances in a single configuration file and spin up a fully configured Elastic deployment on Elastic Cloud.
The Elastic Stack, as code
The Elastic Stack Terraform provider allows you to manage crucial Elasticsearch resources (and in the future, Elastic Stack resources) like:
- Index templates
- Snapshot repositories, snapshot policies
- ILM policies
- Users
- Roles
Using the Elastic Stack provider with Elastic Cloud
The example below breaks down how to use the Elastic Stack provider to jumpstart and configure an Elastic Cloud deployment. Note that the provider works the same with both Elastic Cloud deployments, as well as on-premises or self-managed Elastic Stack clusters.
You can refer to a previous blog post on Elastic Cloud deployment resources. This post focuses on the Elastic Stack provider, its current capabilities, and partially builds on top of the previous blog’s concepts.
In order to use both the Elastic Cloud provider (to spin up an Elastic Stack on Elastic Cloud) and the Elastic Stack provider, simply use an elasticsearch_connection
block alongside outputs from the ec_deployment
resource.
First set up your deployment using the Elastic Cloud provider (ec), and create it with several data tier configurations:
resource "ec_deployment" "elastic-prod" {
region = "gcp-us-central1"
name = "elastic-prod"
version = data.ec_stack.latest.version
deployment_template_id = "gcp-storage-optimized"
elasticsearch {
topology {
id=”cold”
size=”16g”
}
topology {
id=”hot_content”
size=”16g”
}
topology {
id=”warm”
size=”32g”
}
}
kibana {}
}
As the configuration depicts, we’re spinning up an Elastic Cloud deployment on a Google Cloud us-central1 region, with three data tiers (one for cold, hot, and warm contents).
Next, use the Elastic Stack provider to create an index lifecycle management (ILM) policy, to match your data tiers. Create an ILM policy that will roll over hot indices after one day, move them to the warm tier after seven days, cold tier after 30 days, and finally for deletion after 60 days.
resource "elasticstack_elasticsearch_index_lifecycle" "my_ilm" {
name = "my_ilm_policy"
hot {
min_age = "1h"
rollover {
max_age = "1d"
max_size= "50g"
}
}
warm {
min_age="7d"
}
cold {
min_age="30d"
}
delete {
min_age = "60d"
delete {}
}
elasticsearch_connection {
endpoints = [ec_deployment.elastic-prod.elasticsearch[0].https_endpoint]
username = ec_deployment.elastic-prod.elasticsearch_username
password = ec_deployment.elastic-prod.elasticsearch_password
}
}
Use the elasticsearch_connection
to specify the connection to your newly created (and dependent) Elastic Cloud deployment.
The final result
In the example above you’ve used the new Elastic Stack Terraform provider to configure an ILM policy for your newly spun-up Elastic Cloud deployment, all in a single Terraform HCL file.
View the Terraform documentation for more details about configuring and managing your Elastic Stack provider.
To start using the Elastic Cloud today, simply login to the Elastic Cloud console or sign up for a free 14-day trial.