Product

Elastic Cloud Terraform provider now available in beta

By
Omer Kushmaro

We’re excited to share that the official Elastic Cloud Terraform provider is now available in beta. 

Operations and SRE teams often rely on Terraform to safely manage production-related infrastructure using methodologies such as infrastructure as code, which allows you to apply peer-reviewed infrastructure changes in an automated and controlled fashion.

The provider works with Elasticsearch Service on Elastic Cloud, Elastic Cloud Enterprise, and Elasticsearch Service Private environments.

Now you can use the same automation code across public and private clouds, as well as on-premises, with the Terraform provider. 

Getting started

To get started with the beta version of the provider, include the following code block in one of your Terraform .tf configuration files: 

terraform {
  required_version = ">= 0.12"

  required_providers {
    ec = {
      source  = "elastic/ec"
      version = "0.1.0-beta"
    }
  }
}

The provider will be downloaded from the Terraform registry once you initialize it using terraform init.

Note that in order to use the provider, you’ll need to use Terraform CLI version 0.12 or above.

Elastic Cloud resources, as code

The provider allows you to manage Elastic Cloud deployments for your applications and solutions as code, which introduces some exciting use cases.

This initial provider release allows you to manage and deploy Elastic Cloud deployments, and traffic filters as Terraform resources. It also enables you to query Elastic Cloud APIs as Terraform data sources for the latest available Elastic Stack versions and existing cloud deployments.

The Elastic Cloud deployment resource, named ec_deployment in the provider, enables you to specify complex stack configurations, including the number of nodes, their roles, and the amount of memory each node will consume. You can also add or remove solution components like Enterprise Search or APM, and provide each component with its YAML configuration file.

Integrating the Elastic Cloud provider in your CI/CD workflow

A great reason for using Terraform is that you can quickly spin up Elastic Cloud deployments for CI/CD workflows, so you can test and deploy your applications.

Let’s dive into how would you spin up and initialize a deployment with a newly created user and index. The full example code is available for you to use in the Elastic Cloud Terraform Provider GitHub repository.

Create and initialize a deployment

To create our new deployment, we’ll first configure our provider and specify the API key we’d like to authenticate with. The preferred way to do so is to export the environment variable EC_API_KEY with the API key contents as its value. 

provider "ec" { 
  apikey = "API Key Value" 
}

Next, we’ll define our deployment in HashiCorp Configuration Language. For this, we’ll use the “ec_deplyoment” resource. 

resource "ec_deployment" "ci-deployment" {
  name               = "ci-deployment"
  region                 = "us-east-1"
  version                = "7.10.0"
  deployment_template_id = "aws-io-optimized"
  elasticsearch {}
  kibana {}
}

Elastic Cloud is a multi-cloud service, meaning you can deploy resources in the cloud provider and region of your choice. For every deployment resource, you can specify the name and version, too. For more information on available cloud regions, you can refer to our documentation.

You can also use elasticsearch, kibana, enterprise search, and apm blocks to determine the components enabled in your deployment and apply further configurations like node types and memory size per node. This is done by using the topology block of each component, which also defines things like the number of used availability zones or elasticsearch.yml file used for the Elasticsearch cluster settings.

For this example deployment, we’ll keep the elasticsearch and kibana blocks with their default values, which are inferred from the deployment template. You can read more about the specific parameters for an Elastic Cloud deployment in our Elastic Cloud Terraform Provider GitHub repository.

Let’s say you want to perform a few more tricks with your Elastic Cloud deployment, such as creating a few indices and credentials so users can ingest data more quickly.

For this, we’ll use a simple bash script with cURL commands and use some of the output variables we get from our Elastic Cloud deployments: 

output "elasticsearch_https_endpoint" { 
  value = ec_deployment.ci-deployment.elasticsearch[0].https_endpoint 
} 
output "elasticsearch_password" { 
  value = ec_deployment.ci-deployment.elasticsearch_password 
}

The above output code will write important data like your Elasticsearch password and endpoint URL to your terminal. Terraform allows you to use ec_deployment runtime data as values for various things, like a bash script.

To be able to run our bash code, we’ll use both a null resource (a resource meant to run local scripts) and a file template data source to replace bash values with variables containing information about our deployment: 

resource "null_resource" "bootstrap-elasticsearch" { 
  provisioner "local-exec" { 
    command = data.template_file.elasticsearch-configuration.rendered 
  } 
} 
data "template_file" "elasticsearch-configuration" { 
  template   = file("es_config.sh") 
  depends_on = [ec_deployment.ci-deployment] 
  vars = { 
    # Created servers and appropriate AZs 
    elastic-user     = ec_deployment.ci-deployment.elasticsearch_username 
    elastic-password = ec_deployment.ci-deployment.elasticsearch_password 
    es-url           = ec_deployment.ci-deployment.elasticsearch[0].https_endpoint 
  } 
}

The above template file code will use the ec_deployment runtime variable to replace key parts of our bash script code with actual values from the Elastic Cloud deployment created by Terraform.

The null resource is instructed to run the es_config.sh script after it renders it with the “template” values.

To run & test the code for this example, copy the full example files locally, set the provider.tf file with your Elastic Cloud API key and apply the configuration using terraform apply.

The final result

Now you have a complete deployment on Elastic Cloud that’s initialized by the bash script, calling Elasticsearch APIs with any index, user, index templates, or mappings you may want to have when working with your deployment.

Feel free to submit feedback and feature requests directly to our provider repository.

To start using the Elastic Cloud, simply log in to the Elastic Cloud console or sign up for a free 14-day trial.