Herding Llama 3.1 with Elastic and LM Studio

The latest LM Studio 0.3 update has made Elastic’s AI Assistant for Security run with an LM Studio-hosted model easier and faster. In this blog, Elastic and LM Studio teams will show you how to get started in minutes. You no longer need to set up a proxy if you work on the same network or locally on your machine.

Once installed, download your first model by navigating to the Discover tab. Here, you can search for any LLM from Hugging Face and see a list of popular models curated by LM Studio. We will be setting up the Meta Llama 3.1 8B model for Elastic AI Assistant to leverage.

After you’ve picked your model, head to the Developer tab. Here you’ll start a Local Server for the Elastic AI Assistant to connect to. This allows applications or devices on your local network to make API calls to whatever model you run. Requests and responses follow OpenAI's API format.

Click Start Server, then select Llama 3.1 8B from the dropdown menu. You’ll want to ensure the Port is configured to 1234 (it should be like this automatically). Finally, turn on Serve on Local Network. This will allow any device on your network to connect to LM Studio and call whatever model you run.

Now, in the security section, land on one of the subpages to open up the Elastic AI Assistant. Select the gear icon at the top right, and under the Conversation tab, turn streaming off. 

Voilà! You can now use the model you previously loaded in LM Studio to power the Elastic AI Assistant and accomplish great things!

This blog outlined how to easily elevate every security practitioner with Elastic AI Assistant configured with a locally hosted Llama 3.1 Model using LM Studio. This augments the security analysts' expertise by providing AI-powered guidance through triage, investigation, and response, improving productivity. 

To learn more, check out our recent addition to Elastic Security documentation on how to set up Elastic and LM Studio. For further information, refer to:

• Elastic Search AI Platform for SecOps

• Elastic Security | AI Assistant Demo

• Elastic Security SIEM

• LM Studio Documentation

• LM Studio 0.3.0 Update

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

In this blog post, we may have used or referred to third party generative AI tools, which are owned and operated by their respective owners. Elastic does not have any control over the third party tools and we have no responsibility or liability for their content, operation or use, nor for any loss or damage that may arise from your use of such tools. Please exercise caution when using AI tools with personal, sensitive or confidential information. Any data you submit may be used for AI training or other purposes. There is no guarantee that information you provide will be kept secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative AI tools prior to use. 

Elastic, Elasticsearch, ESRE, Elasticsearch Relevance Engine and associated marks are trademarks, logos or registered trademarks of Elasticsearch N.V. in the United States and other countries. All other company and product names are trademarks, logos or registered trademarks of their respective owners.