Querying Elasticsearch
editQuerying Elasticsearch
editBy default, Vega’s data element
can use embedded and external data with a "url" parameter. Kibana adds support for the direct Elasticsearch queries by overloading
the "url" value.
Here is an example of an Elasticsearch query that counts the number of documents in all indexes. The query uses @timestamp field to filter the time range, and break it into histogram buckets.
// An object instead of a string for the url value
// is treated as a context-aware Elasticsearch query.
url: {
// Filter the time picker (upper right corner) with this field
%timefield%: @timestamp
// Apply dashboard context filters when set
%context%: true
// Which indexes to search
index: _all
// The body element may contain "aggs" and "query" subfields
body: {
aggs: {
time_buckets: {
date_histogram: {
// Use date histogram aggregation on @timestamp field
field: @timestamp
// interval value will depend on the daterange picker
// Use an integer to set approximate bucket count
interval: { %autointerval%: true }
// Make sure we get an entire range, even if it has no data
extended_bounds: {
min: { %timefilter%: "min" }
max: { %timefilter%: "max" }
}
// Use this for linear (e.g. line, area) graphs
// Without it, empty buckets will not show up
min_doc_count: 0
}
}
}
// Speed up the response by only including aggregation results
size: 0
}
}
The full result has this kind of structure:
{
"aggregations": {
"time_buckets": {
"buckets": [{
"key_as_string": "2015-11-30T22:00:00.000Z",
"key": 1448920800000,
"doc_count": 28
}, {
"key_as_string": "2015-11-30T23:00:00.000Z",
"key": 1448924400000,
"doc_count": 330
}, ...
Note that "key" is a unix timestamp, and can be used without conversions by the
Vega date expressions.
For most graphs we only need the list of the bucket values, so we use format: {property: "aggregations.time_buckets.buckets"} expression to focus on just the data we need.
Query may be specified with individual range and dashboard context as
well. This query is equivalent to "%context%": true, "%timefield%": "@timestamp",
except that the timerange is shifted back by 10 minutes:
{
body: {
query: {
bool: {
must: [
// This string will be replaced
// with the auto-generated "MUST" clause
"%dashboard_context-must_clause%"
{
range: {
// apply timefilter (upper right corner)
// to the @timestamp variable
@timestamp: {
// "%timefilter%" will be replaced with
// the current values of the time filter
// (from the upper right corner)
"%timefilter%": true
// Only work with %timefilter%
// Shift current timefilter by 10 units back
shift: 10
// week, day (default), hour, minute, second
unit: minute
}
}
}
]
must_not: [
// This string will be replaced with
// the auto-generated "MUST-NOT" clause
"%dashboard_context-must_not_clause%"
]
}
}
}
}
The "%timefilter%" can also be used to specify a single min or max
value. As shown above, the date_histogram’s extended_bounds can be set
with two values - min and max. Instead of hardcoding a value, you may
use "min": {"%timefilter%": "min"}, which will be replaced with the
beginning of the current time range. The shift and unit values are
also supported. The "interval" can also be set dynamically, depending
on the currently picked range: "interval": {"%autointerval%": 10} will
try to get about 10-15 data points (buckets).