About the author
Benjamin Trent is a Lucene committer and member of the project management committee at The Apache Software Foundation and a software engineer at Elastic, where he works on improving Elasticsearch.
Author’s articles
March 20, 2025
Scaling late interaction models in Elasticsearch - part 2
This article explores techniques for making late interaction vectors ready for large-scale production workloads, such as reducing disk space usage and improving computation efficiency.
March 18, 2025
Searching complex documents with ColPali - part 1
The article introduces the ColPali model, a late-interaction model that simplifies the process of searching complex documents with images and tables, and discusses its implementation in Elasticsearch.
February 27, 2025
Filtered HNSW search, fast mode
Explore the improvements we have made for HNSW vector search in Apache Lucene through our ACORN-1 algorithm implementation.
February 7, 2025
Concurrency bugs in Lucene: How to fix optimistic concurrency failures
Thanks to Fray, a deterministic concurrency testing framework from CMU’s PASTA Lab, we tracked down a tricky Lucene bug and squashed it
January 6, 2025
Optimized Scalar Quantization: Even Better Binary Quantization
Here we explain optimized scalar quantization in Elasticsearch and how we used it to improve Better Binary Quantization (BBQ).
December 27, 2024
Lucene bug adventures: Fixing a corrupted index exception
Sometimes, a single line of code takes days to write. Here, we get a glimpse of an engineer's pain and debugging over multiple days to fix a potential Apache Lucene index corruption.
November 18, 2024
Better Binary Quantization (BBQ) vs. Product Quantization
Why we chose to spend time working on Better Binary Quantization (BBQ) instead of product quantization in Lucene and Elasticsearch.
November 11, 2024
Better Binary Quantization (BBQ) in Lucene and Elasticsearch
How Better Binary Quantization (BBQ) works in Lucene and Elasticsearch.
August 27, 2024
Looking back: Elastic's vector search improvements in Elasticsearch & Lucene
Looking back at Elastic's vector search innovations in Elasticsearch and Lucene.
July 17, 2024
Bit vectors in Elasticsearch
Discover what are bit vectors, their practical implications and how to use them in Elasticsearch.
April 26, 2024
Making Elasticsearch and Lucene the best vector database: up to 8x faster and 32x efficient
Discover the recent enhancements and optimizations that notably improve vector search performance in Elasticsearch & Lucene vector database.
April 25, 2024
Understanding Int4 scalar quantization in Lucene
This blog explains how int4 quantization works in Lucene, how it lines up, and the benefits of using int4 quantization.
April 25, 2024
Scalar quantization optimized for vector databases
Optimizing scalar quantization for the vector database use case allows us to achieve significantly better performance for the same retrieval quality at high compression ratios.
December 7, 2023
Introducing kNN Query: An expert way to do kNN search
Explore how the kNN query in Elasticsearch can be used and how it differs from top-level kNN search, including examples.
November 11, 2023
Understanding scalar quantization in Lucene
Explore how Elastic introduced scalar quantization into Lucene, including automatic byte quantization, quantization per segment & performance insights.
October 25, 2023
Scalar quantization 101
Understand what scalar quantization is, how it works and its benefits. This guide also covers the math behind quantization and examples.
September 1, 2023
Bringing maximum-inner-product into Lucene
Explore how we brought maximum-inner-product into Lucene and the investigations undertaken to ensure its support.
August 24, 2023
Adding passage vector search to Lucene
Here's how to add passage vectors to Lucene, the benefits of doing so and how existing Lucene structures can be used to create an efficient retrieval experience.
January 23, 2023
Save space with byte-sized vectors
Elasticsearch is introducing a new type of vector that has 8-bit integer dimensions. This is 4x smaller than the current vector with 32-bit float dimensions, which can result in substantial space savings.
April 20, 2022
Aggregate data faster with new the random_sampler aggregation
Aggregate billions of documents in milliseconds instead of minutes with Elastic. Learn more about how the new random_sampler aggregation gives you statistically robust results at a lower cost.