Remote clusters

The remote clusters functionality enables you to establish unidirectional connections to a remote cluster. Remote clusters are required for cross-cluster replication and cross-cluster search.

Remote cluster connections work by configuring a remote cluster and connecting to a limited number of nodes in that remote cluster. There are two modes for remote cluster connections: sniff mode and proxy mode.

All the communication required between different clusters goes through the transport layer. Remote cluster connections consist of uni-directional connections from the coordinating node to the remote remote connections.

In sniff mode, a cluster is created using a name and a list of seed nodes. When a remote cluster is registered, its cluster state is retrieved from one of the seed nodes and up to three gateway nodes are selected as part of remote cluster requests. This mode requires that the gateway node’s publish addresses are accessible by the local cluster.

Sniff mode is the default connection mode.

The gateway nodes selection depends on the following criteria:

In proxy mode, a cluster is created using a name and a single proxy address. When a remote cluster is registered, a configurable number of socket connections are opened to the proxy address. The proxy is required to route those connections to the remote cluster. Proxy mode does not require remote cluster nodes to have accessible publish addresses.

The proxy mode is not the default connection mode and must be configured. Similar to the sniff gateway nodes, the remote connections are subject to the same version compatibility rules as Rolling upgrades.

You can configure remote clusters globally by using cluster settings, which you can update dynamically. Alternatively, you can configure them locally on individual nodes by using the elasticsearch.yml file.

If you specify the settings in elasticsearch.yml files, only the nodes with those settings can connect to the remote cluster. In other words, functionality that relies on remote cluster requests must be driven specifically from those nodes. For example:

cluster:
    remote:
        cluster_one: 
            seeds: 127.0.0.1:9300 
            transport.ping_schedule: 30s 
        cluster_two: 
            mode: sniff 
            seeds: 127.0.0.1:9301 
            transport.compress: true 
            skip_unavailable: true 
        cluster_three: 
            mode: proxy 
            proxy_address: 127.0.0.1:9302 

For more information about the optional transport settings, see Transport.

If you use cluster settings, the remote clusters are available on every node in the cluster. For example:

PUT _cluster/settings
{
  "persistent": {
    "cluster": {
      "remote": {
        "cluster_one": {
          "seeds": [
            "127.0.0.1:9300"
          ],
          "transport.ping_schedule": "30s"
        },
        "cluster_two": {
          "mode": "sniff",
          "seeds": [
            "127.0.0.1:9301"
          ],
          "transport.compress": true,
          "skip_unavailable": true
        },
        "cluster_three": {
          "mode": "proxy",
          "proxy_address": "127.0.0.1:9302"
        }
      }
    }
  }
}

You can dynamically update the compression and ping schedule settings. However, you must re-include seeds or proxy_address in the settings update request. For example:

PUT _cluster/settings
{
  "persistent": {
    "cluster": {
      "remote": {
        "cluster_one": {
          "seeds": [
            "127.0.0.1:9300"
          ],
          "transport.ping_schedule": "60s"
        },
        "cluster_two": {
          "mode": "sniff",
          "seeds": [
            "127.0.0.1:9301"
          ],
          "transport.compress": false
        },
        "cluster_three": {
          "mode": "proxy",
          "proxy_address": "127.0.0.1:9302",
          "transport.compress": true
        }
      }
    }
  }
}

A remote cluster can be deleted from the cluster settings by setting its settings to null :

PUT _cluster/settings
{
  "persistent": {
    "cluster": {
      "remote": {
        "cluster_two": { 
          "mode": null,
          "seeds": null,
          "skip_unavailable": null,
          "transport": {
            "compress": null
          }
        }
      }
    }
  }
}

These settings apply to both sniff mode and proxy mode. Sniff mode settings and proxy mode settings are described below.

You can use the remote cluster info API to retrieve information about the configured remote clusters, as well as the remote nodes that the node is connected to.