Search and retrieve `semantic_text` fields

Serverless Stack 9.0.0

This page provides instructions for searching and retrieving data from semantic_text fields. Learn how to query semantic_text fields, retrieve indexed chunks, retrieve field embeddings, and highlight the most relevant fragments from search results.

Query `semantic_text` fields

You can query semantic_text fields using the following query types:

Match query: The recommended method for querying semantic_text fields. You can use Query DSL or ES|QL syntax. To learn how to run match queries on semantic_text fields, refer to this example.
kNN query: Finds the nearest vectors to a query vector using a similarity metric, mainly for advanced or combined search use cases. You can use Query DSL or Stack 9.2.0 ES|QL syntax. To learn how to run knn queries on semantic_text fields, refer to this example.
Sparse vector query: Executes searches using sparse vectors generated by a sparse retrieval model such as ELSER. You can use it with Query DSL syntax. To learn how to run sparse vector queries on semantic_text fields, refer to this example.
Semantic query: We don't recommend this legacy query type for new projects, because the alternatives in this list enable more flexibility and customization. The semantic query remains available to support existing implementations.

Retrieve indexed chunks

Serverless Stack 9.2.0

You can retrieve the individual chunks generated by your semantic field's chunking strategy using the fields parameter:

						POST test-index/_search
					{
  "query": {
    "ids" : {
      "values" : ["1"]
    }
  },
  "fields": [
    {
      "field": "semantic_text_field",
      "format": "chunks"
    }
  ]
}
		
	

Use "format": "chunks" to return the field's text as the original text chunks that were indexed.

Retrieve `semantic_text` field embeddings

By default, embeddings generated for semantic_text fields are stored internally and not included in the response when retrieving documents. Retrieving embeddings is useful when you want to:

Reindex documents into another index with the same inference_id without re-running inference
Export or migrate documents while preserving their embeddings
Inspect or debug the raw embeddings generated for your content

The method for retrieving embeddings depends on your Elasticsearch version:

If you use Elasticsearch 9.2 and later, or Serverless, use the _source.exclude_vectors parameter to include embeddings in _source.
If you use Elasticsearch versions earlier than 9.2, use the fields parameter with _inference_fields to retrieve embeddings.

Include embeddings in `_source`

Serverless Stack 9.2.0

To include the full inference fields, including their embeddings, in _source, set the _source.exclude_vectors option to false:

						POST my-index/_search
					{
  "_source": {
    "exclude_vectors": false
  },
  "query": {
    "match_all": {}
  }
}
		
	

The embeddings will appear under _inference_fields in _source.

This works with the Get, Search, and Reindex APIs.

Example: Reindex while preserving embeddings

Serverless Stack 9.2.0

When reindexing documents with semantic_text fields, you can preserve existing embeddings by including them in the source documents. This allows documents to be re-indexed without triggering inference again.

Warning

The target index's semantic_text field must use the same inference_id as the source index to reuse existing embeddings. If the inference_id values do not match, the documents will fail the reindex task.

						POST _reindex
					{
  "source": {
    "index": "my-index-src",
    "_source": {
      "exclude_vectors": false
    }
  },
  "dest": {
    "index": "my-index-dest"
  }
}
		
	

Sends the source documents with their stored embeddings to the destination index.

Example: Troubleshoot `semantic_text` fields

Serverless Stack 9.2.0

To verify that your embeddings look correct or debug embedding generation, retrieve the stored embeddings:

						POST test-index/_search
					{
  "_source": {
    "exclude_vectors": false
  },
  "query": {
    "match": {
      "my_semantic_field": "Which country is Paris in?"
    }
  }
}
		
	

This will return verbose chunked embeddings content that is used to perform semantic search for semantic_text fields:

		{
  "took": 18,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": { "value": 1, "relation": "eq" },
    "max_score": 16.532316,
    "hits": [
      {
        "_index": "test-index",
        "_id": "1",
        "_score": 16.532316,
        "_source": {
          "my_semantic_field": "Paris is the capital of France.",
          "_inference_fields": {
            "my_semantic_field": {
              "inference": {
                "inference_id": ".elser-2-elasticsearch",
                "model_settings": {
                  "service": "elasticsearch",
                  "task_type": "sparse_embedding"
                },
                "chunks": {
                  "my_semantic_field": [
                    {
                      "start_offset": 0,
                      "end_offset": 31,
                      "embeddings": {
                        "airport": 0.12011719,
                        "brussels": 0.032836914,
                        "capital": 2.1328125,
                        "capitals": 0.6386719,
                        "capitol": 1.2890625,
                        "cities": 0.78125,
                        "city": 1.265625,
                        "continent": 0.26953125,
                        "country": 0.59765625,
                        ...
                      }
                    }
                  ]
                }
              }
            }
          }
        }
      }
    ]
  }
}
		
	

The inference endpoint used to generate embeddings.
Lists details about the model used to generate embeddings, such as the service name and task type.
The embeddings generated for this chunk.

Retrieve embeddings using `fields`

Important

This method for returning semantic field embeddings is recommended only for Elasticsearch versions earlier than 9.2. For version 9.2 and later, use the exclude_vectors parameter instead.

To retrieve stored embeddings, use the fields parameter with _inference_fields. This lets you include the vector data that is not shown by default in the response.

						POST my-index/_search
					{
  "query": {
    "match": {
      "my_semantic_field": "Which country is Paris in?"
    }
  },
  "fields": [
    "_inference_fields"
  ]
}
		
	

The fields parameter works with the Search API.

Highlight the most relevant fragments

Extract the most relevant fragments from a semantic_text field using the highlight parameter in the Search API.

Highlight semantic_text fields

Use the highlight parameter with number_of_fragments and order to control fragment selection and sorting:

						POST test-index/_search
					{
    "query": {
        "match": {
            "my_semantic_field": "Which country is Paris in?"
        }
    },
    "highlight": {
        "fields": {
            "my_semantic_field": {
                "number_of_fragments": 2,
                "order": "score"
            }
        }
    }
}
		
	

Specifies the maximum number of fragments to return.
Sorts the most relevant highlighted fragments by score when set to score. By default, fragments are output in the order they appear in the field (order: none).

Enforce semantic highlighter

To restrict highlighting to the semantic highlighter and return no fragments when the field is not of type semantic_text, explicitly set the highlighter type to semantic:

						POST test-index/_search
					{
    "query": {
        "match": {
            "my_field": "Which country is Paris in?"
        }
    },
    "highlight": {
        "fields": {
            "my_field": {
                "type": "semantic",
                "number_of_fragments": 2,
                "order": "score"
            }
        }
    }
}
		
	

Ensures that highlighting is applied exclusively to semantic_text fields.

Retrieve fragments in original order

To retrieve all fragments from the semantic highlighter in their original indexing order without scoring, use a match_all query as the highlight_query:

						POST test-index/_search
					{
  "query": {
    "ids": {
      "values": ["1"]
    }
  },
  "highlight": {
    "fields": {
      "my_semantic_field": {
        "number_of_fragments": 5,
        "highlight_query": { "match_all": {} }
      }
    }
  }
}
		
	

Returns the first 5 fragments. Increase this value to retrieve additional fragments.

Performing cross-cluster search (CCS) with `semantic_text`

Serverless Unavailable Stack 9.2.0

semantic_text supports cross-cluster search (CCS) through the _search endpoint when ccs_minimize_roundtrips is set to true. This is the default value, so most CCS queries work automatically.

Query semantic_text fields across clusters using the standard search endpoint with cluster notation:

						POST local-index,remote-cluster:remote-index/_search
					{
  "query": {
    "match": {
      "my_semantic_field": "Which country is Paris in?"
    }
  }
}
		
	

Search and retrieve semantic_text fields

Search and retrieve `semantic_text` fields