Inner hits

edit

Added in 1.5.0.

This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

The parent/child and nested features allow the return of documents that have matches in a different scope. In the parent/child case, parent document are returned based on matches in child documents or child document are returned based on matches in parent documents. In the nested case, documents are returned based on matches in nested inner objects.

In both cases, the actual matches in the different scopes that caused a document to be returned is hidden. In many cases, it’s very useful to know which inner nested objects (in the case of nested or children or parent documents), or (in the case of parent/child) caused certain information to be returned. The inner hits feature can be used for this. This feature returns per search hit in the search response additional nested hits that caused a search hit to match in a different scope.

Inner hits can be used by defining a inner_hits definition on a nested, has_child or has_parent query and filter. The structure looks like this:

"<query>" : {
    "inner_hits" : {
        <inner_hits_options>
    }
}

If _inner_hits is defined on a query that supports it then each search hit will contain a inner_hits json object with the following structure:

"hits": [
     {
        "_index": ...,
        "_type": ...,
        "_id": ...,
        "inner_hits": {
           "<inner_hits_name>": {
              "hits": {
                 "total": ...,
                 "hits": [
                    {
                       "_type": ...,
                       "_id": ...,
                       ...
                    },
                    ...
                 ]
              }
           }
        },
        ...
     },
     ...
]

Options

edit

Inner hits support the following options:

from

The offset from where the first hit to fetch for each inner_hits in the returned regular search hits.

size

The maximum number of hits to return per inner_hits. By default the top three matching hits are returned.

sort

How the inner hits should be sorted per inner_hits. By default the hits are sorted by the score.

name

The name to be used for the particular inner hit definition in the response. Useful when multiple inner hits have been defined in a single search request. The default depends in which query the inner hit is defined. For has_child query and filter this is the child type, has_parent query and filter this is the parent type and the nested query and filter this is the nested path.

Inner hits also supports the following per document features:

Nested inner hits

edit

The nested inner_hits can be used to include nested inner objects as inner hits to a search hit.

The example below assumes that there is a nested object field defined with the name comments:

{
    "query" : {
        "nested" : {
            "path" : "comments",
            "query" : {
                "match" : {"comments.message" : "[actual query]"}
            },
            "inner_hits" : {} 
        }
    }
}

The inner hit definition in the nested query. No other options need to be defined.

An example of a response snippet that could be generated from the above search request:

...
"hits": {
  ...
  "hits": [
     {
        "_index": "my-index",
        "_type": "question",
        "_id": "1",
        "_source": ...,
        "inner_hits": {
           "comments": { 
              "hits": {
                 "total": ...,
                 "hits": [
                    {
                       "_type": "question",
                       "_id": "1",
                       "_nested": {
                          "field": "comments",
                          "offset": 2
                       },
                       "_source": ...
                    },
                    ...
                 ]
              }
           }
        }
     },
     ...

The name used in the inner hit definition in the search request. A custom key can be used via the name option.

The _nested metadata is crucial in the above example, because it defines from what inner nested object this inner hit came from. The field defines the object array field the nested hit is from and the offset relative to its location in the _source. Due to sorting and scoring the actual location of the hit objects in the inner_hits is usually different than the location a nested inner object was defined.

By default the _source is returned also for the hit objects in inner_hits, but this can be changed. Either via _source filtering feature part of the source can be returned or be disabled. If stored fields are defined on the nested level these can also be returned via the fields feature.

An important default is that the _source returned in hits inside inner_hits is relative to the _nested metadata. So in the above example only the comment part is returned per nested hit and not the entire source of the top level document that contained the the comment.

Hierarchical levels of nested object fields and inner hits.

edit

If a mapping has multiple levels of hierarchical nested object fields each level can be accessed via dot notated path. For example if there is a comments nested field that contains a votes nested field and votes should directly be returned with the the root hits then the following path can be defined:

{
   "query" : {
      "nested" : {
         "path" : "comments.votes",
         "query" : { ... },
         "inner_hits" : {}
      }
    }
}

This indirect referencing is only supported for nested inner hits.

Parent/child inner hits

edit

The parent/child inner_hits can be used to include parent or child

The examples below assumes that there is a _parent field mapping in the comment type:

{
    "query" : {
        "has_child" : {
            "type" : "comment",
            "query" : {
                "match" : {"message" : "[actual query]"}
            },
            "inner_hits" : {} 
        }
    }
}

The inner hit definition like in the nested example.

An example of a response snippet that could be generated from the above search request:

...
"hits": {
  ...
  "hits": [
     {
        "_index": "my-index",
        "_type": "question",
        "_id": "1",
        "_source": ...,
        "inner_hits": {
           "comment": {
              "hits": {
                 "total": ...,
                 "hits": [
                    {
                       "_type": "comment",
                       "_id": "5",
                       "_source": ...
                    },
                    ...
                 ]
              }
           }
        }
     },
     ...

top level inner hits

edit

Besides defining inner hits on query and filters, inner hits can also be defined as a top level construct alongside the query and aggregations definition. The main reason for using the top level inner hits definition is to let the inner hits return documents that don’t match with the main query. Also inner hits definitions can be nested via the top level notation. Other then that the inner hit definition inside the query should be used, because that is the most compact way for defining inner hits.

The following snippet explains the basic structure of inner hits defined at the top level of the search request body:

"inner_hits" : {
    "<inner_hits_name>" : {
        "<path|type>" : {
            "<path-to-nested-object-field|child-or-parent-type>" : {
                <inner_hits_body>
                [,"inner_hits" : { [<sub_inner_hits>]+ } ]?
            }
        }
    }
    [,"<inner_hits_name_2>" : { ... } ]*
}

Inside the inner_hits definition, first the name if the inner hit is defined then whether the inner_hit is a nested by defining path or a parent/child based definition by defining type. The next object layer contains the name of the nested object field if the inner_hits is nested or the parent or child type if the inner_hit definition is parent/child based.

Multiple inner hit definitions can be defined in a single request. In the <inner_hits_body> any option for features that inner_hits support can be defined. Optionally another inner_hits definition can be defined in the <inner_hits_body>.

An example that shows the use of nested inner hits via the top level notation:

{
    "query" : {
        "nested" : {
            "path" : "comments",
            "query" : {
                "match" : {"comments.message" : "[actual query]"}
            }
        }
    },
    "inner_hits" : {
        "comment" : {
            "path" : { 
                "comments" : { 
                    "query" : {
                        "match" : {"comments.message" : "[different query]"} 
                    }
                }
            }
        }
    }
}

The inner hit definition is nested and requires the path option.

The path option refers to the nested object field comments

A query that runs to collect the nested inner documents for each search hit returned. If no query is defined all nested inner documents will be included belonging to a search hit. This shows that it only make sense to the top level inner hit definition if no query or a different query is specified.

Additional options that are only available when using the top level inner hits notation:

path

Defines the nested scope where hits will be collected from.

type

Defines the parent or child type score where hits will be collected from.

query

Defines the query that will run in the defined nested, parent or child scope to collect and score hits. By default all document in the scope will be matched.

Either path or type must be defined. The path or type defines the scope from where hits are fetched and used as inner hits.