Predicting leaks, optimizing resources: how a water utilities company uses Elastic to process millions of transactions daily to power smart water management

Imagine a world where water utilities can predict leaks before they happen, optimize resource allocation in real time, and provide customers with up-to-the-minute usage insights. This company, a century-old innovator in the water utilities sector, isn't just imagining this future — it’s building it. With the help of Elastic, the organization now has data and analytics at its fingertips that help reduce waste and contribute to sustainable use of the world’s most precious resource.

This is a game-changer for the company, which serves water utilities, municipalities, and industrial customers worldwide. "We help customers manage massive volumes of water, and to do so, we handle massive volumes of data," says the director of software engineering. "Elasticsearch and Elastic Observability enable us to address some of the biggest challenges facing our sector — from leak detection to water quality and demand management."

In the early 2010s, the company recognized the potential of wireless communications and developed devices that transmit readings via cellular networks. Customers were equally enthusiastic, fueling demand for devices and software that utilities use to manage their water resources.

Today, approximately one-third of US water utilities use this software, which presents the business with another challenge: how to manage rapid growth in data ingested from millions of devices.

Previously, the company stored data in SQL databases, but this technology wasn't up to the task — not to mention the effort required to expand capacity. Managing data partitioning, multiple writer instances, and scaling read capacity all required extensive manual configuration, putting additional pressure on the engineering team.

When researching the market for a replacement, Elasticsearch stood out for its scalability. The company also became an early adopter of Elastic's time series data streams (TSDS), allowing it to efficiently store and search massive volumes of time-series data and eliminate bottlenecks that had plagued its previous SQL-based system.

Over the years, the company has deployed Elastic Observability to power real-time dashboards for customers. Elastic's aggregation capabilities enable instant status updates on individual services and the overall system. By efficiently aggregating large volumes of structured data, the company can perform system-wide calculations at scale with rapid response times.

Initially, the company managed Elastic deployments on Amazon EC2 instances, handling updates and infrastructure maintenance internally. To reduce this burden, it migrated to Elastic Cloud on AWS, shifting much of its storage infrastructure to a managed service. This freed up its lean engineering and DevOps teams to focus on enhancing customer value through improved dashboards and user interfaces rather than managing infrastructure.

Currently, the company stores 37 terabytes of raw numerical meter reading data on Elastic. Its active shard count is approximately 16,000, and its cluster consists of around 54 instances across various storage tiers — hot, warm, cold, and frozen. Elastic processes roughly 30 million transactions daily, with each transaction containing up to six hours of data, resulting in over a billion data points received from the company’s devices each day.

Today, Elasticsearch and Elastic Observability form the backbone of the company's suite of water management solutions that integrate water technology, software, and services to support data-driven decisions and optimize water operations for utilities, municipalities, and commercial or industrial customers.

Elastic Observability enables the company's clients to analyze their data via interactive dashboards. Utilities can apply filters and tags to generate custom aggregations and charts, gaining deep insights into water consumption patterns. Other benefits include the ability to observe industrial water use or pinpoint high-volume consumers like data centers, helping to detect trends and potential issues.

The Director of Software Engineering says, "When customers receive unexpectedly high water bills, they often dispute the charges. Our system allows utilities to pinpoint the exact day of high usage, facilitating informed conversations and minimizing customer dissatisfaction. Providing this level of detail enhances operational efficiency and improves customer communication."

Another key use case is non-revenue water management. Many utilities accept water loss rates of 15%–20% due to leaks and metering inaccuracies. The US Environmental Protection Agency (EPA) tracks this information, and some states require utilities to report water loss figures. Elastic not only supports accurate reporting, it also accommodates data from the latest generation of pressure monitoring and acoustic leak detection sensors that enable utilities to address potential issues before they escalate.

The company is pioneering the integration of Elasticsearch as a vector database for RAG using OpenAI's large language model (LLM). This allows users to query their data in natural language. For example, they can ask, "What are the potential sources of this leak?" The AI then analyzes usage patterns and identifies possible causes, such as an outdoor leak or a gradual increase in consumption.

"Elastic's Search AI Platform made it easy for us to explore new capabilities with AI and consolidate our infrastructure. Its ecosystem integrations with LangChain and other tools, flexibility to use our own model, and the fact that we don't need separate point solutions all proved to be valuable for our organization. With Elastic, we unified our data sources, making it far easier for AI models to query information accurately," says the Director of Software Engineering. "Our RAG-AI model returns accurate, coherent answers so that end-users spend less time scrolling through links or chunks of information."

Further strengthening its partnership with Elastic, the company has embedded a Designated Support Engineer (DSE) from Elastic Consulting into its account. While still in its early stages, this collaboration is expected to optimize its Elastic deployment and accelerate its innovation roadmap, ensuring continued AI leadership in water management solutions.