Elastic Managed LLMs

Elastic provides built-in LLMs through managed AI connectors.
These connectors are accessed and managed through the Elastic Inference Service (EIS), which is the single entry point for using Elastic Managed LLMs.

• Requires the manage_inference cluster privilege (the built-in inference_admin role grants this privilege)
• For on-premises installations (Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, or self managed clusters), Elastic Managed LLMs are only available through EIS with Cloud Connect. Your Elastic Stack version must be 9.3 or later.

Elastic Managed LLMs are available exclusively through the Elastic Inference Service. You can find the list of supported models on the EIS documentation page.

The Elastic Managed LLMs use third party service providers for inference. Refer to the Elastic Inference Service page for details.

Customer projects or deployments hosted in any cloud service provider or region have access to Elastic Managed LLMs in the AWS US region us-east-1. All data is encrypted in transit. The LLMs are configured for zero data retention: none of the prompts or outputs are stored by the service provider.

Only request metadata is logged in AWS CloudWatch. No information related to prompts is retained. Logged metadata includes the timestamp, model used, region, and request status.

Read more at our AI Data FAQs to learn about our data practices for AI related features.

Elastic Managed LLMs incur a cost per million tokens for input and output tokens. Refer to the Elastic pricing page that correspond to your Elastic setup for details.