How to

Personalizing Elastic App Search with results based on search history

By
Nick Chow

With Elastic App Search, you can add scalable, relevant search experiences to all your apps and websites. It offers a host of search result personalization options out of the box, such as weights and boosts and curations. You could also add a these documents might interest you feature, which would surface additional content for users, similar to documents they’ve previously searched for. This post walks you through the process of creating this capability using the robust App Search APIs.

Building the search client

The search client is built with the frontend application as usual, except for two additional requirements (apart from creating the actual suggestion views):

  • Tag each analytics event with a user ID.
    • For example, for each query and click, you’d send an additional analytics tags parameter:
      • curl -X GET 'https://154d5f7d80774345fg92c8381891faf7.ent-search.us-east-1.aws.cloud.es.io/api/as/v1/engines/national-parks-demo/search' \ 
-H 'Content-Type: application/json' \ 
-H 'Authorization: Bearer search-soaewu2ye6uc45dr8mcd54v8' \ 
-d '{ 
  "query": "everglade", 
  "analytics": { 
    "tags": ["UNIQUE_USER_ID"] 
  } 
}'
  • When a list of the suggested results is needed (search request, page load, etc.), fire a request to the external controller.

Building the external controller

The external controller is the backend service. You would have to build this to generate the query you can use to populate a list of documents based on that user’s past searches. On request, the external controller would need to do the following:

  1. Get the terms the user had searched for previously:
    1. Call the App Search analytics API for a list of the top n queries over m time range filtered by that user ID as a tag. Here’s an example that returns the top 20 queries in the last two months of 2020 for the user tagged UNIQUE_USER_ID.
      2. curl -X GET 'https://154d5f7d80774345fg92c8381891faf7.ent-search.us-east-1.aws.cloud.es.io/api/as/v1/engines/national-parks-demo/analytics/queries' \ 
-H 'Content-Type: application/json' \ 
-H 'Authorization: Bearer private-xxxxxxxxxxxxxxxxxxxxxxxx' \ 
-d '{ 
  "filters": {  
    "all": [ 
      { 
        "date": { 
          "from": "2020-10-31T12:00:00+00:00" 
          "to": "2020-12-31T00:00:00+00:00" 
        } 
      }, { 
        "tag": "UNIQUE_USER_ID" 
      } 
    ] 
  }, 
  "page": { 
    "size": 20 
  } 
}'
  2. (Optional) It’s possible to exclude documents that have been found through search results if, for example, you’d like to promote content or products that are more likely to be new to the user. Find the documents the user had clicked on and to exclude them:
    1. Call the App Search analytics API for a list of the clicked documents filtered by user ID.
  3. Generate suggested documents:
    1. Issue a multiple search query to the App Search search API using the search terms generated from step 1. 
      2. curl -X POST 'https://154d5f7d80774345fg92c8381891faf7.ent-search.us-east-1.aws.cloud.es.io/api/as/v1/engines/national-parks-demo/multi_search' \ 
-H 'Content-Type: application/json' \ 
-H 'Authorization: Bearer search-soaewu2ye6uc45dr8mcd54v8' \ 
-d '{ 
  "queries": [ 
    {"query": "california"}, 
    {"query": "florida"} 
  ] 
}'
    2. (Optional) Add a filter to that query to exclude the documents the user has already clicked on (generated from step 2).
    3. Return the results of that query to the client.

FAQ and additional considerations

Here’s a list of questions and other things to consider as you build.

Can I use this for other segmentation beyond just individual users?

Yes! You can use this in whatever segmentation method you prefer. The tags are the key. Tags use strings that you define, so that could be by user, by geographic region, or any other cohort you can define based on what you know about the user. 

But remember, those tags need to be defined by search event and by click event. If you choose to change it up in the future and you’re not logging that data you’ll need to start over or infer cohorts otherwise.

What if I want to display search results based on some other arbitrary user data I own?

That’s great! As long as you can turn that data into query terms, you can modify the external controller to include those search results, too.

How can I tune this feature?

Outside of your existing relevance tuning settings, you can sharpen the results in a few ways:

  • Limit the number of user queries returned more strictly
  • Limit the time range of user queries returned more strictly
  • Limit the total results returned from the multi-search query

Why can’t I do this on the front end?

You could if your client has a list of the user’s queries and optionally, documents on hand — for example with a cookie. Just don’t forget that private keys are required for analytics API access, which you would never want to expose.

Next steps

If you’d like to experiment with building this search history-based feature, you can spin up a free trial of App Search on Elastic Cloud (or you can download and self-manage). If you have any questions or would like to let us know how your project is going, please drop us a line in our discussion forums.