New

The executive guide to generative AI

Read more
About usPartnersSupport|Login
IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Tutorial: semantic search with ELSER Search across clusters »
Elastic Docs ›Elasticsearch Guide [8.12] ›Search your data ›Semantic search

Tutorial: semantic search with the inference API

edit

Tutorial: semantic search with the inference API

edit

The instructions in this tutorial shows you how to use the inference API with the Open AI service to perform semantic search on your data. The following example uses OpenAI’s text-embedding-ada-002 second generation embedding model. You can use any OpenAI models, they are all supported by the inference API.

Requirements

edit

An OpenAI account is required to use the inference API with the OpenAI service.

Create the inference task

edit

Create the inference task by using the Create inference API:

PUT _inference/text_embedding/openai_embeddings 
{
    "service": "openai",
    "service_settings": {
        "api_key": "<api_key>" 
    },
    "task_settings": {
       "model": "text-embedding-ada-002" 
    }
}
Copy as curlTry in Elastic 

The task type is text_embedding in the path.

The API key of your OpenAI account. You can find your OpenAI API keys in your OpenAI account under the API keys section. You need to provide your API key only once. The Get inference API does not return your API key.

The name of the embedding model to use. You can find the list of OpenAI embedding models here.

Create the index mapping

edit

The mapping of the destination index - the index that contains the embeddings that the model will create based on your input text - must be created. The destination index must have a field with the dense_vector field type to index the output of the OpenAI model.

PUT openai-embeddings
{
  "mappings": {
    "properties": {
      "content_embedding": { 
        "type": "dense_vector", 
        "dims": 1536, 
        "element_type": "float",
        "similarity": "dot_product" 
      },
      "content": { 
        "type": "text" 
      }
    }
  }
}
Copy as curlTry in Elastic 

The name of the field to contain the generated tokens. It must be refrenced in the inference pipeline configuration in the next step.

The field to contain the tokens is a dense_vector field.

The output dimensions of the model. Find this value in the OpenAI documentation of the model you use.

The faster` dot_product` function can be used to calculate similarity because OpenAI embeddings are normalised to unit length. You can check the OpenAI docs about which similarity function to use.

The name of the field from which to create the sparse vector representation. In this example, the name of the field is content. It must be referenced in the inference pipeline configuration in the next step.

The field type which is text in this example.

Create an ingest pipeline with an inference processor

edit

Create an ingest pipeline with an inference processor and use the OpenAI model you created above to infer against the data that is being ingested in the pipeline.

response = client.ingest.put_pipeline(
  id: 'openai_embeddings',
  body: {
    processors: [
      {
        inference: {
          model_id: 'openai_embeddings',
          input_output: {
            input_field: 'content',
            output_field: 'content_embedding'
          }
        }
      }
    ]
  }
)
puts response
PUT _ingest/pipeline/openai_embeddings
{
  "processors": [
    {
      "inference": {
        "model_id": "openai_embeddings", 
        "input_output": { 
          "input_field": "content",
          "output_field": "content_embedding"
        }
      }
    }
  ]
}
Copy as curlTry in Elastic 

The name of the inference model you created by using the Create inference API.

Configuration object that defines the input_field for the inference process and the output_field that will contain the inference results.

Load data

edit

In this step, you load the data that you later use in the inference ingest pipeline to create embeddings from it.

Use the msmarco-passagetest2019-top1000 data set, which is a subset of the MS MARCO Passage Ranking data set. It consists of 200 queries, each accompanied by a list of relevant text passages. All unique passages, along with their IDs, have been extracted from that data set and compiled into a tsv file.

Download the file and upload it to your cluster using the Data Visualizer in the Machine Learning UI. Assign the name id to the first column and content to the second column. The index name is test-data. Once the upload is complete, you can see an index named test-data with 182469 documents.

Ingest the data through the inference ingest pipeline

edit

Create the embeddings from the text by reindexing the data throught the inference pipeline that uses the OpenAI model as the inference model.

response = client.reindex(
  wait_for_completion: false,
  body: {
    source: {
      index: 'test-data',
      size: 50
    },
    dest: {
      index: 'openai-embeddings',
      pipeline: 'openai_embeddings'
    }
  }
)
puts response
POST _reindex?wait_for_completion=false
{
  "source": {
    "index": "test-data",
    "size": 50 
  },
  "dest": {
    "index": "openai-embeddings",
    "pipeline": "openai_embeddings"
  }
}
Copy as curlTry in Elastic 

The default batch size for reindexing is 1000. Reducing size to a smaller number makes the update of the reindexing process quicker which enables you to follow the progress closely and detect errors early.

The rate limit of your OpenAI account may affect the throughput of the reindexing process. If this happens, change size to 3 or a similar value in magnitude.

The call returns a task ID to monitor the progress:

GET _tasks/<task_id>
Copy as curlTry in Elastic 

You can also cancel the reindexing process if you don’t want to wait until the reindexing process is fully complete which might take hours:

POST _tasks/<task_id>/_cancel
Copy as curlTry in Elastic 

Semantic search

edit

After the dataset has been enriched with the embeddings, you can query the data using semantic search. Pass a query_vector_builder to the k-nearest neighbor (kNN) vector search API, and provide the query text and the model you have used to create the embeddings.

If you cancelled the reindexing process, you run the query only a part of the data which affects the quality of your results.

response = client.search(
  index: 'openai-embeddings',
  body: {
    knn: {
      field: 'content_embedding',
      query_vector_builder: {
        text_embedding: {
          model_id: 'openai_embeddings',
          model_text: 'Calculate fuel cost'
        }
      },
      k: 10,
      num_candidates: 100
    },
    _source: [
      'id',
      'content'
    ]
  }
)
puts response
GET openai-embeddings/_search
{
  "knn": {
    "field": "content_embedding",
    "query_vector_builder": {
      "text_embedding": {
        "model_id": "openai_embeddings",
        "model_text": "Calculate fuel cost"
      }
    },
    "k": 10,
    "num_candidates": 100
  },
  "_source": [
    "id",
    "content"
  ]
}
Copy as curlTry in Elastic 

As a result, you receive the top 10 documents that are closest in meaning to the query from the openai-embeddings index sorted by their proximity to the query:

"hits": [
      {
        "_index": "openai-embeddings",
        "_id": "DDd5OowBHxQKHyc3TDSC",
        "_score": 0.83704096,
        "_source": {
          "id": 862114,
          "body": "How to calculate fuel cost for a road trip. By Tara Baukus Mello • Bankrate.com. Dear Driving for Dollars, My family is considering taking a long road trip to finish off the end of the summer, but I'm a little worried about gas prices and our overall fuel cost.It doesn't seem easy to calculate since we'll be traveling through many states and we are considering several routes.y family is considering taking a long road trip to finish off the end of the summer, but I'm a little worried about gas prices and our overall fuel cost. It doesn't seem easy to calculate since we'll be traveling through many states and we are considering several routes."
        }
      },
      {
        "_index": "openai-embeddings",
        "_id": "ajd5OowBHxQKHyc3TDSC",
        "_score": 0.8345704,
        "_source": {
          "id": 820622,
          "body": "Home Heating Calculator. Typically, approximately 50% of the energy consumed in a home annually is for space heating. When deciding on a heating system, many factors will come into play: cost of fuel, installation cost, convenience and life style are all important.This calculator can help you estimate the cost of fuel for different heating appliances.hen deciding on a heating system, many factors will come into play: cost of fuel, installation cost, convenience and life style are all important. This calculator can help you estimate the cost of fuel for different heating appliances."
        }
      },
      {
        "_index": "openai-embeddings",
        "_id": "Djd5OowBHxQKHyc3TDSC",
        "_score": 0.8327426,
        "_source": {
          "id": 8202683,
          "body": "Fuel is another important cost. This cost will depend on your boat, how far you travel, and how fast you travel. A 33-foot sailboat traveling at 7 knots should be able to travel 300 miles on 50 gallons of diesel fuel.If you are paying $4 per gallon, the trip would cost you $200.Most boats have much larger gas tanks than cars.uel is another important cost. This cost will depend on your boat, how far you travel, and how fast you travel. A 33-foot sailboat traveling at 7 knots should be able to travel 300 miles on 50 gallons of diesel fuel."
        }
      },
      (...)
    ]
« Tutorial: semantic search with ELSER Search across clusters »