5 reasons why observability and security work well together

DISH Media's ad revenue business ingests and processes 10 billion records a day from 25 million device endpoints — including operational, business, and security data. With a unified solution and single agent through Elastic, dashboards and data are now available in a single pane of glass across teams for quick analysis.

This has significantly reduced incident detection and MTTR, improving customer experiences. Because of Elastic’s single agent, anomalies across millions of systems and customer devices are spotted much more quickly, accelerating root cause analysis and remediation. And with the team operating from a single agent, there were no additional implementation costs.

“With Elastic, we now have a unified view of data that we can correlate to detect patterns and anomalies,” says John Haskell, head of engineering at DISH Media. “In the past, root cause analysis and remediation could take weeks. Now it takes hours.”

View the full DISH Media story.

OpenTelemetry, one of the highest velocity projects in the Cloud Native Computing Foundation (CNCF) ecosystem, is considered the de facto standard for telemetry data and is a widely adopted framework for both SRE and security teams. OTel Semantic Conventions framework helps users reduce the time and effort required for querying and correlating diverse data, building visualizations, and analyzing features for machine learning applications. 

Normalizing security and observability data with the OTel Semantic Conventions is a powerful tool that drastically reduces the complexity that so often hinders the efficient analysis of software, performance, and security issues. SRE and security teams along with technology vendors are embracing open data standards to enable holistic analysis of diverse and heterogeneous data.

SIEM solutions and other security technologies that integrate with observability platforms leverage insights from logs, metrics, and traces. This unified approach enables the proactive identification of abnormal patterns, suspicious activities, and potential security incidents. 

By correlating abnormal log spikes in network traffic with server performance metrics, organizations can quickly distinguish between legitimate traffic surges and potential DDoS attacks. Unusual patterns, such as repeated login failures or access from unusual locations, are quickly surfaced — significantly reducing the likelihood of successful attacks.

Enterprise cloud data management leader Informatica replaced its complex observability and SIEM solutions with Elastic’s unified platform. It boosted application performance while protecting systems from external threats — saving big on budget in the process.

"With Elastic, we have a single vendor for observability and SIEM. This represents a cost saving of 50% compared to other solutions for an organization of our size," says Amreth Chandrasehar, director of ML engineering, observability, and site reliability engineering at Informatica.

And performance need not be compromised by consolidation. In fact, Informatica found the opposite to be true. "Elastic's search functionality is incredibly fast,” explains Chandrasehar. “We store trillions of documents, but a search query returns accurate results in little more than 10 seconds."

View the full Informatica story.

No industry knows compliance requirements quite like the financial sector. Emirates NBD, one of the largest banking groups in the Middle East by assets, built a centralized logging system that crunches multiple terabytes of data a day from a multitude of data sources. With Elastic at its core, this new environment amounts to what Vice President of Cloud and Data Platforms at Emirates NBD Ali Rey characterizes as the foundation of a single source of truth. 

Centralized logging provides the bank with an avenue to beef up security and to store and retrieve audit logs required by governance stakeholders. "If there are any disputes, or any questions, queries or anything that happens from both an internal or external point of view, we've got these audit logs that haven't been tampered with," Rey says.

Thanks to its migration to centralized logging with Elastic, the bank has expanded from its original observability investment into security, which helps it detect both external and internal threats.

View the full Emirates NBD story.

When observability and security functions work in harmony, they ensure a more secure and reliable operational environment. A fortified security practice is not only an essential defensive measure for business and reputational health, but also a catalyst for the efficacy of observability tools. And, in a self-perpetuating cycle, security posture is further bolstered by discrepancies surfaced from observability monitoring. 

Relying on a unified data platform based on open standards for both security and observability practices may seem like a distant goal, but taking the initial steps today will prepare your organization for the long term. 

Read the SANS report, Shining a light in the dark: Observability + security, or watch the webinar to learn more about this emerging strategy and how you can take steps to unify your organization’s observability and security functions.