ES|QL metadata fields

edit

ES|QL can access metadata fields. The following are currently supported:

  • _index: the index the document belongs to. The field is of the type keyword.
  • _id: the source document’s ID. The field is of the type keyword.
  • _version: the source document’s version. The field is of the type long.
  • _source: the original JSON document body that was passed at index time (or a reconstructed version if synthetic _source_ is enabled). The field is loaded as a special _source type. This field is not supported by functions.

To enable access to these fields, the FROM source command requires a dedicated directive:

FROM index [METADATA _index, _id, _source]

Metadata fields are only available if the source of the data is an index. Consequently, FROM is the only source commands that supports the METADATA directive.

Once enabled, these fields will be available to subsequent processing commands, just like other index fields:

FROM ul_logs, apps [METADATA _index, _version]
| WHERE id IN (13, 14) AND _version == 1
| EVAL key = CONCAT(_index, "_", TO_STR(id))
| SORT id, _index
| KEEP id, _index, _version, key
id:long _index:keyword _version:long key:keyword

13

apps

1

apps_13

13

ul_logs

1

ul_logs_13

14

apps

1

apps_14

14

ul_logs

1

ul_logs_14

Similar to index fields, once an aggregation is performed, a metadata field will no longer be accessible to subsequent commands, unless used as a grouping field:

FROM employees [METADATA _index, _id]
| STATS max = MAX(emp_no) BY _index
max:integer _index:keyword

10100

employees