WARNING: Version 0.90 of Elasticsearch has passed its EOL date.

This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.

« Hadoop Gateway HTTP »
Elastic Docs ›Reference [0.90] ›Modules ›Gateway

S3 Gateway

edit

S3 Gateway

edit

The S3 gateway is deprecated and will be removed in a future version. Please use the local gateway instead.

S3 based gateway allows to do long term reliable async persistency of the cluster state and indices directly to Amazon S3. Here is how it can be configured:

cloud:
    aws:
        access_key: AKVAIQBF2RECL7FJWGJQ
        secret_key: vExyMThREXeRMm/b/LRzEB8jWwvzQeXgjqMX+6br


gateway:
    type: s3
    s3:
        bucket: bucket_name

You’ll need to install the cloud-aws plugin, by running bin/plugin install cloud-aws before (re)starting elasticsearch.

The following are a list of settings (prefixed with gateway.s3) that can further control the S3 gateway:

Setting Description

chunk_size

Big files are broken down into chunks (to overcome AWS 5g limit and use concurrent snapshotting). Default set to 100m.

concurrent_streams

edit

The gateway.s3.concurrent_streams allow to throttle the number of streams (per node) opened against the shared gateway performing the snapshot operation. It defaults to 5.

Region

edit

The cloud.aws.region can be set to a region and will automatically use the relevant settings for both ec2 and s3. The available values are: us-east-1, us-west-1, ap-southeast-1, eu-west-1.

« Hadoop Gateway HTTP »