« reverse state »
Elastic Docs ›Curator Reference [8.0] ›Filter Elements

source

edit

source

edit

The source from which to derive the index or snapshot age. Can be one of name, creation_date, or field_stats.

This setting is only used with the age filtertype, or
with the space filtertype when use_age is set to True.

When using the age filtertype, source requires
direction, unit, unit_count,
and additionally, the optional setting, epoch.

name-based ages

edit

Using name as the source tells Curator to look for a timestring within the index or snapshot name, and convert that into an epoch timestamp (epoch implies UTC).

 - filtertype: age
   source: name
   direction: older
   timestring: '%Y.%m.%d'
   unit: days
   unit_count: 3

A word about regular expression matching with timestrings

Timestrings are parsed from strftime patterns, like %Y.%m.%d, into regular expressions. For example, %Y is 4 digits, so the regular expression for that looks like \d{4}, and %m is 2 digits, so the regular expression is \d{2}.

What this means is that a simple timestring to match year and month, %Y.%m will result in a regular expression like this: ^.*\d{4}\.\d{2}.*$. This pattern will match any 4 digits, followed by a period ., followed by 2 digits, occurring anywhere in the index name. This means it will match monthly indices, like index-2016.12, as well as daily indices, like index-2017.04.01, which may not be the intended behavior.

To compensate for this, when selecting indices matching a subset of another pattern, use a second filter with exclude set to True

- filtertype: pattern
 kind: timestring
 value: '%Y.%m'
- filtertype: pattern
 kind: timestring
 value: '%Y.%m.%d'
 exclude: True

This will prevent the %Y.%m pattern from matching the %Y.%m part of the daily indices.

This applies whether using timestring as a mere pattern match, or as part of date calculations.

creation_date-based ages

edit

creation_date extracts the epoch time of index or snapshot creation.

 - filtertype: age
   source: creation_date
   direction: older
   unit: days
   unit_count: 3

field_stats-based ages

edit

source can only be field_stats when filtering indices.

In Curator 5.3 and older, source field_stats uses the Field Stats API to calculate either the min_value or the max_value of the field as the stats_result, and then use that value for age comparisons. In 5.4 and above, even though it is still called field_stats, it uses an aggregation to calculate the same values, as the field_stats API is no longer used in Elasticsearch 6.x and up.

field must be of type date in Elasticsearch.

 - filtertype: age
   source: field_stats
   direction: older
   unit: days
   unit_count: 3
   field: '@timestamp'
   stats_result: min_value
« reverse state »