Build and buy: Why a durable enterprise architecture delivers business impact at scale

A durable AI architecture is what enables the “build and buy” strategy to function. It allows teams to move from simple, low-complexity AI use cases to highly complex agentic workflows. The architecture has two main areas of focus: context retrieval and performance monitoring.

The key to high-value AI is ensuring that AI models have deep, accurate, and secure context about your business by providing access to high-quality data. Context retrieval mechanisms are built using context engineering, a collection of practices and capabilities that enable large language models (LLMs) to access the right information to complete a desired task. 

1. Enterprise knowledge from SaaS applications: Ingest, unify, and store unstructured, semi-structured, and structured business data at scale from SaaS applications and enterprise systems via integrations and connectors within an open vector database. This includes additional information, such as standard operating procedures, company policies, and business processes, that enable AI responses to be grounded in business logic and proprietary data. 

2. Information security: Implement and enforce data security on enterprise knowledge using document- and field-level security and role-based access controls (RBAC) so that only the right users have access to relevant information. 

3. Search retrieval: For LLMs to pull the most relevant context, teams need access to advanced search techniques that can match user query intent with relevant context from source systems. This includes various search and ranking techniques — semantic search with machine learning (ML), hybrid retrieval, advanced relevance tuning, and reranking — to power AI applications that deliver accurate results.

4. AI/ML integrations: With a platform approach, enterprise IT teams can quickly access preferred LLMs and AI development frameworks to speed up development. Using ML models, teams can further tune relevance to pass the most relevant context to LLMs, improving accuracy, user experience, and trust in outputs. 

The more context that’s provided, the more accurate the responses and actions become. Based on the level of use-case complexity, adding context will increase the effort of context engineering and architectural complexity. However, the upfront investment to build an architecture that pulls distributed data and context from source systems will alleviate the technical burden as more use cases are built on the architecture.

Without AI performance management, IT teams have no visibility and can’t trust or drive adoption of AI experiences. Layering observability into your architecture is a crucial pillar in the “build and buy” strategy. 

1. Application performance monitoring (APM): APM provides critical insights, including response times, error rates, and resource utilization, to help track the performance of the services and infrastructure supporting the AI application. By tracking bottlenecks and anomalies, teams can quickly isolate issues and resolve potential incidents before they impact customer experience.

2. LLM observability: LLM observability provides critical business insights into AI model performance, context accuracy, and usage patterns, such as user engagement, response accuracy, conversation quality, and user sentiment. Grouping these conversations and interactions by use case provides additional insight into how users are engaging. These insights are essential signals for developers, enabling them to quickly fine-tune experiences, expand successful use cases, and augment solutions aligned with real user needs that further enhance user satisfaction and accelerate adoption. 

3. Cloud monitoring: Most AI applications are hosted in the cloud because workflows are resource-intensive and dynamic. This allows for easier access to enterprise-ready LLMs and AI frameworks, autoscaling infrastructure, application availability based on SLAs/SLOs, and cost management. With cloud monitoring, teams can correlate the performance and cost of cloud infrastructure, enabling them to quickly diagnose infrastructure-related bottlenecks and optimize resource usage. 

With observability tools, development teams can easily track the performance and accuracy of AI applications.

The “build and buy” approach is necessary to shift from short-term project thinking to a long-term AI strategy. Existing SaaS investments can and should be used to provide AI experiences with all the relevant context needed to produce accurate outputs. By prioritizing an architecture-first approach, your organization can move beyond fragmented AI features and build toward a scalable AI solution.  

At Elastic, we build our durable enterprise AI architecture using our Search AI Platform. We use Elasticsearch to build context-retrieval mechanisms via context engineering, providing LLMs with the most relevant data and tools so that our AI experiences deliver accurate responses and actions. And with direct access to Elastic Observability within the platform, we can monitor technology performance to deliver reliable AI applications that drive business impact. 

Learn how we built a durable enterprise architecture for Support Assistant on Elastic’s Search AI Platform.