WARNING: Version 5.4 of the Elastic Stack has passed its EOL date.
This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.
Alerting on Cluster and Index Events
editAlerting on Cluster and Index Events
editX-Pack alerting is a set of administrative features that enable you to watch for changes or anomalies in your data and perform the necessary actions in response. For example, you might want to:
- Monitor social media as another way to detect failures in user-facing automated systems like ATMs or ticketing systems. When the number of tweets and posts in an area exceeds a threshold of significance, notify a service technician.
- Monitor your infrastructure, tracking disk usage over time. Open a helpdesk ticket when any servers are likely to run out of free space in the next few days.
- Track network activity to detect malicious activity, and proactively change firewall configuration to reject the malicious user.
- Monitor Elasticsearch, and send immediate notification to the system administrator if nodes leave the cluster or query throughput exceeds an expected range.
- Track application response times and if page-load time exceeds SLAs for more than 5 minutes, open a helpdesk ticket. If SLAs are exceeded for an hour, page the administrator on duty.
All of these use-cases share a few key properties:
- The relevant data or changes in data can be identified with a periodic Elasticsearch query.
- The results of the query can be checked against a condition.
- One or more actions are taken if the condition is true — an email is sent, a 3rd party system is notified, or the query results are stored.
How Watches Work
editX-Pack provides an API for creating, managing and testing watches. A watch describes a single alert and can contain multiple notification actions.
A watch is constructed from four simple building blocks:
- Schedule
- A schedule for running a query and checking the condition.
- Query
- The query to run as input to the condition. Watches support the full Elasticsearch query language, including aggregations.
- Condition
- A condition that determines whether or not to execute the actions. You can use simple conditions (always true), or use scripting for more sophisticated scenarios.
- Actions
- One or more actions, such as sending email, pushing data to 3rd party systems through a webhook, or indexing the results of the query.
A full history of all watches is maintained in an Elasticsearch index. This history keeps track of each time a watch is triggered and records the results from the query, whether the condition was met, and what actions were taken.
Watches run with elevated privileges. Users mapped to the built-in
watcher_admin role or any other role to which the manage_watcher cluster
privilege is assigned should be reviewed and granted only to personnel with
appropriate trust levels to read and write all indices.