IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Upgrade to data streams Upgrade to legacy versions of Elastic APM »
Elastic Docs › ›Upgrade

Switch to the Elastic APM integration

edit

Switch to the Elastic APM integration

edit

Why switch?

edit

The new APM ecosystem offers a number of benefits:

Fleet:

  • A single, unified way to add monitoring for logs, metrics, traces, and other types of data to each host — install one thing instead of multiple
  • Central, unified configuration management — no need to edit multiple configuration files

Data streams:

  • Reduced number of fields per index, better space efficiency, and faster queries
  • More granular data control
  • Errors and metrics data streams are shared with other data sources — which means better long-term integration with the logs and metrics apps
  • Removes template inheritance for ILM policies and makes use of new Elasticsearch index and component templates
  • Fixes resource 'apm-7.16.3-$type' exists, but it is not an alias

APM Integration:

  • Easier to install APM on edge machines
  • Improved source map handling and APM agent configuration management
  • Less configuration
  • Easier and less error-prone upgrade path
  • Zero-downtime configuration changes

APM integration architecture

edit

The Elastic APM integration consists of four components: APM agents, the Elastic APM integration, Elasticsearch, and Kibana. Generally, there are two ways that these four components can work together:

APM agents on edge machines send data to a centrally hosted APM integration:

Architecture of Elastic APM

Or, APM agents and the APM integration live on edge machines and enroll via a centrally hosted Elastic Agent:

Architecture of Elastic APM option two

In order to collect data from RUM and mobile agents, which run in browser and mobile applications, you must run Elastic Agent centrally. For other applications, such as backend services, Elastic Agent may be co-located on the edge machine.

Limitations

edit

There are some limitations to be aware of:

  • This change cannot be reverted
  • Currently, only the Elasticsearch output is supported
  • APM runs under Elastic Agent which, depending on the installation method, might be require root privileges
  • An Elastic Agent with the APM integration enabled must be managed by Fleet.

Switch a self-managed cluster

edit
  1. Upgrade the Elastic Stack
  2. Add a Fleet Server
  3. Install a Fleet-managed Elastic Agent
  4. Add the APM integration
  5. Stop the legacy APM Server

Upgrade the Elastic Stack

edit

The Elastic Stack (Elasticsearch and Kibana) must be upgraded to version 7.14 or higher. See the Elastic Stack Installation and Upgrade Guide for guidance.

Review the APM release notes, breaking changes, and Observability What’s new content for important changes between your current APM version and this one.

Add a Fleet Server

edit

Fleet Server is a component of the Elastic Stack used to centrally manage Elastic Agents. The APM integration requires a Fleet Server to be running and accessible to your hosts. Add a Fleet Server by following this guide.

If you’re upgrading a self-managed deployment of the Elastic Stack, you’ll need to enable Elasticsearch security and the API key service.

After adding your Fleet Server host and generating a service token, the in-product help in Kibana will provide a command to run to start an Elastic Agent as a Fleet Server. Commands may require administrator privileges.

Verify Fleet Server is running by navigating to Fleet > Agents in Kibana.

Install a Fleet-managed Elastic Agent

edit

It’s possible to install the Elastic APM integration on the same Elastic Agent that is running the Fleet Server integration. For this use case, skip this step.

The Fleet-managed Elastic Agent will run the Elastic APM integration on your edge nodes, next to your applications. To install a Fleet-managed Elastic Agent, follow this guide.

Add the APM integration

edit

The APM integration receives performance data from your APM agents, validates and processes it, and then transforms the data into Elasticsearch documents.

To add the APM integration, see Step 3: Add the APM integration. Only complete the linked step (not the entire quick start guide). If you’re adding the APM integration to a Fleet-managed Elastic Agent, you can use the default policy. If you’re adding the APM integration to the Fleet Server, use the policy that the Fleet Server is running on.

You’ll configure the APM integration in this step. See Input settings for a reference of all available settings. As long as the APM integration is configured with the same secret token or you have API keys enabled on the same host, no reconfiguration is required in your APM agents.

Stop the legacy APM Server

edit

Once data from upgraded APM agents is visible in the APM app, it’s safe to stop the legacy APM Server process.

Congratulations — you now have the latest and greatest in Elastic APM!

Switch an Elastic Cloud cluster

edit
  1. Upgrade the Elastic Stack
  2. Switch to Elastic Agent
  3. Configure the APM integration
  4. Scale APM and Fleet

Upgrade the Elastic Stack

edit

Use the Elastic Cloud console to upgrade the Elastic Stack to version 7.16.3. See the Elasticsearch Service upgrade guide for details.

Switch to Elastic Agent

edit

APM data collection will be interrupted while the migration is in progress. The process of migrating should only take a few minutes.

With a Superuser account, complete the following steps:

  1. In Kibana, navigate to Observability > APM > Settings > Schema.

    switch to Elastic Agent

  2. Click Switch to Elastic Agent. Make a note of the apm-server.yml user settings that are incompatible with Elastic Agent. Check the confirmation box and click Switch to Elastic Agent.

    Elastic Agent settings migration

Elastic Cloud will now create a Fleet Server instance to contain the new APM integration, and then will shut down the old APM server instance. Within minutes your data should begin appearing in the APM app again.

Configure the APM integration

edit

You can now update settings that were removed during the upgrade. See Input settings for a reference of all available settings.

In Kibana, navigate to Mangement > Fleet. Select the Elastic Cloud Agent Policy. Next to the Elastic APM integration, select Actions > Edit integration.

Scale APM and Fleet

edit

Certain Elasticsearch output configuration options are not available with the APM integration. To ensure data is not lost, you can scale APM and Fleet up and out. APM’s capacity to process events increases with the instance memory size.

Go to the Elastic Cloud console, select your deployment and click Edit. Here you can edit the number and size of each availability zone.

scale apm

Congratulations -- you now have the latest and greatest in Elastic APM!

« Upgrade to data streams Upgrade to legacy versions of Elastic APM »