Shield with Marvel

Marvel consists of a user interface over a data exporter known as the agent. The agent runs on each node and accesses that node’s monitoring API. The agent can store this collected data locally, on the cluster, or send the data to an external monitoring cluster. Users can view and analyze the collected data with the Marvel UI.

To work with the Shield plugin, Marvel’s configuration needs to be adapted for the production cluster, which is the cluster being monitored, as well as the monitoring cluster, where the monitoring data is stored. For clusters that store their own monitoring data, apply both sets of settings to the single, production cluster.

You will configure at least two users to work with Marvel. These users have to exist on the monitoring cluster.

The Agent needs to be assigned a user with the correct privileges to write data to the Marvel indices named .marvel-*, check the Marvel index template, and upload the Marvel index template. You need only one agent user.
Marvel UI users must authenticate and have privileges to read data from the Marvel indices. These users also need to able to call the Nodes Info API in order to get the monitoring cluster’s Elasticsearch version. This version check allows Marvel to be compatible with many versions of Elasticsearch. You can have as many of these end users configured as you would like.

The default roles.yml file includes definitions for these two roles. The steps below show you how to create these users on the monitoring cluster.

Monitoring Cluster Settings

edit

The monitoring cluster is used to both store and view the Marvel data. When configuring Shield, you need to perform the following actions:

Make sure there is a user created with the marvel_agent role. Marvel uses this to export the data.
Make sure there is a user created with the marvel_user role. You use this to view the Marvel UI and get license information.
When using Marvel on a production server, you must enter your Marvel License. This license is stored in the monitoring cluster. This step needs to be performed once, by a user with permissions to write to the .marvel-kibana index. The .marvel-kibana index is used to store Marvel UI settings (for example, set custom warning levels) and therefore write permission for .marvel-kibana is required for UI customizations. Both storing license and storing settings can be done by any user added to the marvel_user role.

This is in the default roles.yml

marvel_agent:
  cluster: indices:admin/template/get, indices:admin/template/put
  indices:
    '.marvel-*': indices:data/write/bulk, create_index

marvel_user:
  cluster: cluster:monitor/nodes/info, cluster:admin/plugin/license/get
  indices:
    '.marvel-*': all

Once the roles are configured, create a user for the agent:

bin/shield/esusers useradd marvel_export -p strongpassword -r marvel_agent

Then create one or more users for the Marvel UI:

bin/shield/esusers useradd USER -p strongerpassword -r marvel_user

Production Cluster Settings

edit

The Marvel agent is installed on every node in the production cluster. The agent collects monitoring data from the production cluster and stores the data on the monitoring cluster. The agent’s configuration specifies a list of hostname and port combinations for access to the monitoring cluster.

When the monitoring cluster uses the Shield plugin and is configured to accept only HTTPS requests, you must configure the agent on the production cluster to use HTTPS instead of the default HTTP protocol.

Authentication and protocol configuration are both controlled by the marvel.agent.exporter.es.hosts setting in the node’s elasticsearch.yml file. The setting accepts a list of monitoring cluster servers to serve as a fallback in case a server is unavailable. Each of these servers must be properly configured, as in the following example:

Example marvel.agent.exporter.es.hosts setting

marvel.agent.exporter.es.hosts: [ "https://USER:PASSWORD@node01:9200", "https://USER:PASSWORD@node02:9200"]

When the monitoring cluster uses HTTPS, the Marvel agent will attempt to validate the certificate of the Elasticsearch node in the monitoring cluster. If you are using your own CA you should specify a trust store that has the signing certificate of the CA. Here is an example config for the marvel.agent.exporter.es.truststore.* settings:

marvel.agent.exporter.es.hosts: [ "https://USER:PASSWORD@node01:9200", "https://USER:PASSWORD@node02:9200"]
marvel.agent.exporter.es.ssl.truststore.path: FULL_FILE_PATH
marvel.agent.exporter.es.ssl.truststore.password: PASSWORD

See the Marvel documentation for more details about other SSL related settings.

The 1.3.0 release of Marvel adds HTTPS support.

Marvel user interface & Sense

edit

The Marvel UI supports SSL without the need for any additional configuration. You can change URL access scheme for Marvel to HTTPS.

Users attempting to access the Marvel UI with the URL https://HOST:9200/_plugin/marvel must provide valid credentials. See Monitoring Cluster settings for information on the required user configuration.

Sense also supports HTTPS access. Users that access Sense over URLs of the form https://host:9200/_plugin/marvel/sense/index.html must provide valid credentials if they have not already authenticated to a dashboard.

Users connecting to the production cluster with Sense must provide valid credentials. Clusters must be configured to enable cross-origin requests to enable users to connect with Sense. See the CORS documentation for details.

Providing user credentials to Sense in order to access another cluster is only supported in releases 1.3.0 and later of Marvel.

« Shield with Logstash Kibana »