Top 5 ways to optimize your Elastic Cloud costs

Elastic Cloud is the best and most cost effective way to consume Elastic. If you’re already using or are considering Elastic Cloud, here are some recommendations on optimizing costs.

Elastic Cloud uses resource-based pricing, where you pay for the infrastructure resources consumed to run the Elastic Stack capabilities for your use case. Therefore, the best way to optimize Elastic Cloud costs is to streamline resource consumption. 

You can reduce your resource consumption by leveraging our global presence and several cost-saving capabilities. Here are the top five recommendations: 

[Related article: How does Elastic Cloud pricing work?]

1) Use searchable snapshots and data tiers to optimize storage costs

You can optimize storage, search performance, and depth of insight from the data in your Elasticsearch cluster by using searchable snapshots and data tiers. 

Elastic searchable snapshots let you search across data in snapshots stored on low-cost object storage, such as AWS S3, Microsoft Azure Storage, and Google Cloud Storage.

With data tiers, you can select between hot, warm, cold, and frozen tiers to optimize storage cost and query performance based on the needs of your use case. You can store data that you access the most in the expensive hot tier and search it in milliseconds. On the other hand, if part of your data is less frequently accessed and you can afford query times of seconds or minutes, you can store that data in the cheaper warm, cold, or frozen tiers. The result: you get greater control of your costs based on the value of your data. 

Use of data tiers, especially cold and frozen tiers, can help significantly reduce infrastructure costs with minimal performance impact.

2) Upgrade to 7.15 release or higher to reduce data transfer costs

A part of the data transfer and storage (DTS) cost arises from data transfer inter-node traffic. Starting from the 7.15 release, we compress data transfer inter-node indexing traffic and use snapshot storage for data relocation and recovery. This significantly reduces the inter-node traffic and the associated data transfer costs. 

These capabilities benefit Elastic Cloud customers with heavy indexing or data migration between data tiers. For example, at Elastic, we were able to reduce our own inter-node traffic by more than 50% from 464 TB/day to 204.5 TB/day, resulting in $1.2 million/year savings. 

[Related article: Reduce data transfer and storage (DTS) costs in Elastic Cloud]

3) Use the right hardware profile and machine types

Many organizations are looking for ways to improve price-performance across their use cases.As an AWS Graviton Ready Partner, we are at the forefront of supporting the AWS Graviton2 powered instance types for Elastic Cloud deployments. 

If you are on AWS, you can benefit from our support for hardware profiles that utilize the Graviton2 ARM-based variants of M5 and C5 instance types, namely the M6gd and the C6gd VMs, in regions where they are available. This allows you to gain more than 20% improvement in price-performance over comparable Intel chipsets. 

Use our Elasticsearch Service AWS instance configurations doc, Elasticsearch Service AWS instance configurations, to choose the right instance configuration for your workload.

If you are on GCP, you can leverage our support for Google Compute Engine (GCE) N2 general purpose virtual machine (VM) types, and additional hardware configuration options powered by N2 custom machine types. N2 machine types provide a balance of compute, memory, and storage and also offer more than a 20% improvement in price-performance over the first-generation N1 machines. See Introducing new Google Cloud instance type on Elastic Cloud for more details.

Similarly, Elastic has expanded support for Microsoft Azure Virtual Machine (VM) types on Elastic Cloud. You can now select new hardware profiles backed by Edsv4, Ddv4, and Fsv2 VM series for deployments in Azure in all supported Azure regions. Read here to choose the right Azure instance configuration for your workload.

Finally, Elastic continues to add support for new hardware profiles to provide better price-performance on Elastic Cloud. With the 8.3 release, Elastic provides hardware profile migration capability, using which you can switch between hardware profiles on your existing deployment. This allows you to take advantage of more efficient machine types provided by different cloud providers. Read Improving performance and optimizing cost on Elastic Cloud has never been easier for more information.

4) Use auto scaling policies

Furthermore, you can define policy for auto scaling and optimize cost and management. Your Elastic cluster will scale as your usage expands, and you will not run out of disk. Via the autoscaling API or GUI prompt, an Elasticsearch cluster can report whether or not it needs additional resources to meet the policy. For example, you can define a policy that a warm tier should scale on available disk space and scale-up the data nodes accordingly. 

5) Start with Cloud monthly billing to come up with the optimal configuration

The configuration for self-managed deployment and Elastic Cloud can be different. Your optimal configuration for Elastic Cloud, and consequent pricing, depends on your actual workload. If you're not sure of the optimal configuration, we recommend that you start with Elastic Cloud monthly billing and configure the resources to adequately support your workload. 

Understand your deployment, how your data and consumption is growing, and the associated cost. You can start by signing up for a free trial. Or, you can contact us to come up with an optimal configuration, and associated price, for your use case. 

As you scale and increase the amount of data processed through Elasticsearch, you can leverage these recommendations to better manage your costs. One of the key advantages of using Elastic is that your costs do not have to increase proportionately with your usage.