Hadoop Gateway

edit

The hadoop gateway is deprecated and will be removed in a future version. Please use the local gateway instead.

The hadoop (HDFS) based gateway stores the cluster meta and indices data in hadoop. Hadoop support is provided as a plugin and installing is explained here or downloading the hadoop plugin and placing it under the plugins directory. Here is an example config to enable it:

gateway:
    type: hdfs
    hdfs:
        uri: hdfs://myhost:8022

Settings

edit

The hadoop gateway requires two simple settings. The gateway.hdfs.uri controls the URI to connect to the hadoop cluster, for example: hdfs://myhost:8022. The gateway.hdfs.path controls the path under which the gateway will store the data.

concurrent_streams

edit

The gateway.hdfs.concurrent_streams allow to throttle the number of streams (per node) opened against the shared gateway performing the snapshot operation. It defaults to 5.