IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« S3 Repository Plugin S3 Repository »
Elastic Docs ›Elasticsearch Plugins and Integrations [5.4] ›Snapshot/Restore Repository Plugins ›S3 Repository Plugin

Getting started with AWS

edit
IMPORTANT: This documentation is no longer updated. Refer to Elastic's version policy and the latest documentation.

Getting started with AWS

edit

The plugin will default to using IAM Role credentials for authentication. These can be overridden by, in increasing order of precedence, system properties aws.accessKeyId and aws.secretKey, environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_KEY, or the elasticsearch config using cloud.aws.access_key and cloud.aws.secret_key or if you wish to set credentials specifically for s3 cloud.aws.s3.access_key and cloud.aws.s3.secret_key:

cloud:
    aws:
        access_key: AKVAIQBF2RECL7FJWGJQ
        secret_key: vExyMThREXeRMm/b/LRzEB8jWwvzQeXgjqMX+6br

Transport security

edit

By default this plugin uses HTTPS for all API calls to AWS endpoints. If you wish to configure HTTP you can set cloud.aws.protocol in the elasticsearch config. You can optionally override this setting per individual service via: cloud.aws.ec2.protocol or cloud.aws.s3.protocol.

cloud:
    aws:
        protocol: https
        s3:
            protocol: http
        ec2:
            protocol: https

In addition, a proxy can be configured with the proxy.host, proxy.port, proxy.username and proxy.password settings (note that protocol can be http or https):

cloud:
    aws:
        protocol: https
        proxy:
            host: proxy1.company.com
            port: 8083
            username: myself
            password: theBestPasswordEver!

You can also set different proxies for ec2 and s3:

cloud:
    aws:
        s3:
            proxy:
                host: proxy1.company.com
                port: 8083
                username: myself1
                password: theBestPasswordEver1!
        ec2:
            proxy:
                host: proxy2.company.com
                port: 8083
                username: myself2
                password: theBestPasswordEver2!

Region

edit

The cloud.aws.region can be set to a region and will automatically use the relevant settings for both ec2 and s3. You can specifically set it for s3 only using cloud.aws.s3.region. The available values are:

  • us-east (us-east-1) for US East (N. Virginia)
  • us-east-2 for US East (Ohio)
  • us-west (us-west-1) for US West (N. California)
  • us-west-2 for US West (Oregon)
  • ap-south (ap-south-1) for Asia Pacific (Mumbai)
  • ap-southeast (ap-southeast-1) for Asia Pacific (Singapore)
  • ap-southeast-2 for Asia Pacific (Sydney)
  • ap-northeast (ap-northeast-1) for Asia Pacific (Tokyo)
  • ap-northeast-2 (ap-northeast-2) for Asia Pacific (Seoul)
  • eu-west (eu-west-1) for EU (Ireland)
  • eu-west-2 (eu-west-2) for EU (London)
  • eu-central (eu-central-1) for EU (Frankfurt)
  • sa-east (sa-east-1) for South America (São Paulo)
  • cn-north (cn-north-1) for China (Beijing)
  • ca-central (ca-central-1) for Canada (Central)

S3 Signer API

edit

If you are using a S3 compatible service, they might be using an older API to sign the requests. You can set your compatible signer API using cloud.aws.signer (or cloud.aws.s3.signer) with the right signer to use.

If you are using a compatible S3 service which do not support Version 4 signing process, you may need to use S3SignerType, which is Signature Version 2.

Read timeout

edit

Read timeout determines the amount of time to wait for data to be transferred over an established, open connection before the connection is timed out. Defaults to AWS SDK default value (50s). It can be configured with cloud.aws.read_timeout (or cloud.aws.s3.read_timeout) setting:

cloud.aws.read_timeout: 30s
« S3 Repository Plugin S3 Repository »