S3 Gateway

edit

The S3 gateway is deprecated and will be removed in a future version. Please use the local gateway instead.

S3 based gateway allows to do long term reliable async persistency of the cluster state and indices directly to Amazon S3. Here is how it can be configured:

cloud:
    aws:
        access_key: AKVAIQBF2RECL7FJWGJQ
        secret_key: vExyMThREXeRMm/b/LRzEB8jWwvzQeXgjqMX+6br


gateway:
    type: s3
    s3:
        bucket: bucket_name

You’ll need to install the cloud-aws plugin, by running bin/plugin install cloud-aws before (re)starting elasticsearch.

The following are a list of settings (prefixed with gateway.s3) that can further control the S3 gateway:

Setting Description

chunk_size

Big files are broken down into chunks (to overcome AWS 5g limit and use concurrent snapshotting). Default set to 100m.

concurrent_streams

edit

The gateway.s3.concurrent_streams allow to throttle the number of streams (per node) opened against the shared gateway performing the snapshot operation. It defaults to 5.

Region

edit

The cloud.aws.region can be set to a region and will automatically use the relevant settings for both ec2 and s3. The available values are: us-east-1, us-west-1, ap-southeast-1, eu-west-1.