« Elastic Cloud Static IPs Monitoring your deployment »
Elastic Docs ›Elasticsearch Service Documentation ›Securing your deployment

Encrypt your deployment with a customer-managed encryption key

edit

Encrypt your deployment with a customer-managed encryption key

edit

By default, Elastic already encrypts your deployment data and snapshots at rest. You can reinforce this mechanism by providing your own encryption key, also known as Bring Your Own Key (BYOK). To do that, you need a customer-managed key that you set up and manage in your cloud provider’s Key Management Service (KMS).

Encryption at rest using customer-managed keys is only available for the Enterprise subscription level, when creating new deployments. The ability to edit encryption settings for existing deployments will be supported at a later date.

Using a customer-managed key allows you to strengthen the security of your deployment data and snapshot data at rest. Note that if you use a custom snapshot repository different from the one provided by Elastic Cloud, these snapshots are not encrypted with your customer-managed key by default. The encryption happens at the file system level.

How using a customer-managed key helps to improve your data security

edit

Using a customer-managed key helps protect against threats related to the management and control of encryption keys. It does not directly protect against any specific types of attacks or threats. However, the ability to keep control over your own keys can help mitigate certain types of threats such as:

  • Insider threats. By using a customer-managed key, Elastic does not have access to your encryption keys [1]. This can help prevent unauthorized access to data by insiders with malicious intent.
  • Compromised physical infrastructure. If a data center is physically compromised, the hosts are shut off. With customer-managed key encryption, that’s a second layer of protection that any malicious intruder would have to bypass, in addition to the existing built-in hardware encryption.

Using a customer-managed key can help comply with regulations or security requirements, but it is not a complete security solution by itself. There are other types of threats that it does not protect against.

[1] You set up your customer-managed keys and their access in your key management service. When you provide a customer-managed key identifier to Elastic Cloud, we do not access or store the cryptographic material associated with that key. Customer-managed keys are not directly used to encrypt deployment or snapshot data. Elastic Cloud accesses your customer-managed keys to encrypt and decrypt data encryption keys, which, in turn, are used to encrypt the data.

When a deployment encrypted with a customer-managed key is deleted or terminated, its data is locked first before being deleted, ensuring a fully secure deletion process.

Prerequisites

edit

Know before you go

edit

At this time, the following features are not supported:

Create an encryption key for your deployment

edit

  1. Create a symmetric single-region key or multi-region replica key. The key must be available in each region in which you have deployments to encrypt. You can use the same key to encrypt multiple deployments. Later, you will need to provide the Amazon Resource Name (ARN) of that key or key alias to Elastic Cloud.

    Use an alias ARN instead of the key ARN itself if you plan on doing manual key rotations. When using a key ARN directly, only automatic rotations are supported.

  2. Apply a key policy with the settings required by Elastic Cloud to the key created in the previous step:

    {
  "Sid": "ElasticKeyAccess",
  "Effect": "Allow",
  "Principal": {
    "AWS": "*"
  },
  "Action": [
    "kms:Decrypt", 
    "kms:Encrypt", 
    "kms:GetKeyRotationStatus", 
    "kms:GenerateDataKey", 
    "kms:DescribeKey" 
  ],
  "Resource": "*",
  "Condition": { 
    "ForAnyValue:StringLike": {
      "aws:PrincipalOrgPaths": "o-ygducmlz12/r-e5t3/ou-e5t3-fzpdq76p/ou-e5t3-ysfcmd95/ou-e5t3-hwt05su3/*"
   }
 }
}

    kms:Decrypt - This operation is used to decrypt data encryption keys stored on the deployment’s host, as well as decrypting snapshots stored in S3.

    kms:Encrypt - This operation is used to encrypt the data encryption keys generated by the KMS as well as encrypting your snapshots.

    kms:GetKeyRotationStatus - This operation is used to determine whether automatic key rotation is enabled.

    kms:GenerateDataKey - This operation is used to generate a data encryption key along with an encrypted version of it. The system leverages the randomness provided by the KMS to produce the data encryption key and your actual customer-managed key to encrypt the data encryption key.

    kms:DescribeKey - This operation is used to check whether your key is properly configured for Elastic Cloud. In addition, Elastic Cloud uses this to check if a manual key rotation was performed by comparing underlying key IDs associated with an alias.

    This condition allows the accounts associated with the Elastic Cloud production infrastructure to access your key. Under typical circumstances, Elastic Cloud will only be accessing your key via two AWS accounts: the account your deployment’s host is in and the account your S3 bucket containing snapshots is in. However, determining these particular account IDs prior to the deployment creation is not possible at the moment. This encompasses all of the possibilities. For more on this, check the AWS documentation.

Create a deployment encrypted with your key

edit

  1. Create a new deployment. You can do it from the Elasticsearch Service Console, or from the API:

    • from the Elasticsearch Service Console:

      • Select Create deployment from the Elasticsearch Service Console home page.
      • In the Settings, set the Cloud provider to Amazon Web Services and select a region.
      • Expand the Advanced settings and turn on Use a customer-managed encryption key. An additional field appears to let you specify the ARN of the AWS KMS key or key alias you will use to encrypt your new deployment.
      • Configure the rest of your deployment to your convenience, and select Create deployment.

    • using the API:

      • Choose a cloud region and a deployment template (also called hardware profile) for your deployment from the list of available regions, deployment templates, and instance configurations.
      • Get a valid Elastic Cloud API key with the Organization owner role or the Admin role on deployments. These roles allow you to create new deployments.
      • Get the ARN of the symmetric AWS KMS key or of its alias. Use an alias if you are planning to do manual key rotations as specified in the AWS documentation.

      • Use these parameters to create a new deployment with the Elastic Cloud API. For example:

        curl -XPOST \
-H 'Content-Type: application/json' \
-H "Authorization: ApiKey <replace with encoded API key>" \
"https://api.elastic-cloud.com/api/v1/deployments?template_id=<replace with desired template ID>" \
-d '
{
  "name": "my-deployment",
  "version": "8.15.0",
  "region": "us-east-1",
  "settings": {
    "byok": {
      "key_resource_path": "<replace with your key or alias ARN>"
    }
  }
}

        You can also create the deployment from a snapshot of a deployment that was initially not encrypted with a customer-managed key. You can use this as a workaround to encrypt existing data under new deployments using your key, until encrypting existing deployments with a customer-managed key is supported.

The deployment is now created and encrypted using the specified key. Future snapshots will also be encrypted using that key.

You can check that your hosted deployment is correctly encrypted with the key you specified. To do that, go to the deployment’s Security page and select Manage encryption key in Encryption at rest.

Rotate a customer-managed key

edit

Elastic Cloud will automatically rotate the keys every 31 days as a security best practice.

You can also trigger a manual rotation in AWS KMS, which will take effect in Elastic Cloud within 30 minutes. For manual rotations to work, you must use an alias when creating the deployment. We do not currently support on-demand rotations but plan on supporting this in the future.

Revoke a customer-managed key

edit

Revoking a customer-managed key in your key management service can be a break-glass procedure in case of a security breach. Elastic Cloud gets an error if an encryption key is disabled, deleted, or if the appropriate role is removed from the IAM policy. Within 30 minutes maximum, Elastic Cloud locks the directories in which your deployment data live and prompts you to delete your deployment as an increased security measure.

If that happens and this is not intended, you can restore the key in the key management system. Your deployment operations will resume when the key can be reached again. For more details, check Troubleshooting.

When a customer-managed key is permanently revoked and isn’t restored, the data stored in Elastic Cloud is effectively crypto-shredded.

In a future release of Elastic Cloud, you will be able to:

  • Remove a customer-managed key and revert your deployment to using an Elastic-managed encryption.
  • Edit the customer-managed key in use in a deployment to re-encrypt it with a different key.

Encrypt an existing deployment using a new customer-managed key

edit

Encrypting deployments with a customer-managed key is currently only possible for new deployments. In a future release of Elastic Cloud, you will be able to:

  • Encrypt an existing Elastic Cloud deployment with a customer-managed key.
  • Edit the customer-managed key in use in a deployment to re-encrypt it with a different key.

Troubleshooting

edit

My deployment became inaccessible. What’s causing this?

When Elastic Cloud can’t reach the encryption key, your deployment may become inaccessible. The most common reasons for this issue are:

  • Connectivity issues between Elastic Cloud and the KMS.

    When Elastic Cloud is unable to access the customer-managed key, Elastic is alerted and will work to identify the cause. Elastic does not pause or terminate deployment instances when detecting connectivity issues, but your deployment may be inaccessible until issues are fixed.

  • The customer-managed key was deleted or revoked on the KMS.

    Restore or recover your key, and if need be, rotate your key and associate a new key before deleting your old key. Elastic Cloud will send you alerts prompting you to restore the key if it cannot access your key and your deployment is not operational.

    Within 30 minutes maximum, Elastic Cloud locks the directories in which your deployment data live and prompts you to delete your deployment as an increased security measure.

    While it is locked, the deployment retains all data but is not readable or writable*:

    • If access to the key is never restored, the deployment data does not become accessible again

    • When restoring access to the key, the deployment becomes operational again:

      • If Elastic didn’t have to perform any platform operations on your instances during the locked period, operations are restored with minimum downtime.
      • If Elastic performed some platform operations on your instances during the locked period, restoring operations can require some downtime. It’s also possible that some data can’t be restored** depending on the available snapshots.

*During the locked directory period, Elastic may need to perform platform operations on the machines hosting your instances that result in data loss on the Elasticsearch data nodes but not the deployment snapshots.

**Elastic recommends that you keep snapshots of your deployment in custom snapshot repositories in your own CSP account for data recovery purposes.

« Elastic Cloud Static IPs Monitoring your deployment »