Upgrade Elastic Security

edit

To upgrade from 7.16.0 or earlier to 8.7.1, you must first upgrade to 7.17, which enables you to use the Upgrade Assistant to prepare for the upgrade. Before you upgrade, you must resolve all critical issues identified by the Upgrade Assistant.

Rolling upgrades are unsupported in Elastic Security, which runs within the Kibana application. To upgrade, you must shut down all Kibana instances, install the new software, and restart Kibana. Upgrading while older Kibana instances are running can cause data loss or upgrade failures.

When required, Kibana automatically migrates saved objects. In case of an upgrade failure, you can roll back to an earlier version of Kibana. To roll back, you must have a backup snapshot that includes the kibana feature state. By default, snapshots include the kibana feature state.

For more information about upgrading, refer to Upgrade to Elastic 8.7.1.

You can upgrade to pre-release versions for testing, but upgrading from a pre-release to the Generally Available version is unsupported. You should use pre-release versions only for testing in a temporary environment.

Upgrading multiple Kibana instances

edit

When upgrading several Kibana instances connected to the same Elasticsearch cluster, ensure that all outdated instances are shut down before starting the upgrade.

Rolling upgrades are unsupported in Kibana. However, when outdated instances are shut down, you can start all upgraded instances in parallel, which allows all instances to participate in the upgrade migration in parallel.

For large deployments with more than 10 Kibana instances, and more than 10,000 saved objects, you can reduce the upgrade downtime by bringing up a single Kibana instance and waiting for it to complete the upgrade migration before bringing up the remaining instances.

You can upgrade to pre-release versions for testing, but upgrading from a pre-release to the Generally Available version is unsupported. You should use pre-release versions only for testing in a temporary environment.

Support for Elastic prebuilt detection rule automatic updates

edit

Automatic updates of Elastic prebuilt detection rules are supported for the current Elastic Security version and the latest three previous minor releases. For example, if you’re upgrading to Elastic Security 8.10, you’ll be able to use the Rules UI to update your prebuilt rules until Elastic Security 8.14 is released. After that point, you can still manually download and install updated prebuilt rules, but you must upgrade to the latest Elastic Security version to receive automatic updates.

Preparing for migration

edit

Take these extra steps to ensure you are ready for migration.

Ensure your Elasticsearch cluster is healthy

edit

Problems with your Elasticsearch cluster can prevent Kibana upgrades from succeeding.

During the upgrade process, Kibana creates new indices into which updated documents are written. If a cluster is approaching the low watermark, there’s a high risk of Kibana not being able to create these. Reading, transforming, and writing updated documents can be memory intensive, using more available heap than during routine operation. You must make sure that enough heap is available to prevent requests from timing out or throwing errors from circuit breaker exceptions. You should also ensure that all shards are replicated and assigned.

A healthy cluster has:

  • Enough free disk space, at least twice the amount of storage taken up by the .kibana and .kibana_task_manager indices
  • Sufficient heap size
  • A "green" cluster status

Ensure that all Kibana instances are the same

edit

When you perform an upgrade migration of different Kibana versions, the migration can fail. Ensure that all Kibana instances are running the same version, configuration, and plugins.

Back up your data

edit

Be sure to have a snapshot of all your data before attempting a migration. If something goes wrong during migration, you can restore from the snapshot and try again.

Review the common causes of Kibana upgrade failures and how to prevent them.

Upgrade from a 7.x version

edit

Upgrade Elastic Agents

edit

Upgrade your Elastic Stack and Elastic Agents to 7.17 first (refer to Upgrade Fleet-managed Elastic Agents). Afterwards, you can upgrade the Elastic Stack to 8.x. Initially, Elastic Agents will be version 7.17; this is fine because Elastic Security 8.x supports the last minor release in 7.x (7.17) and any subsequent Elastic Endpoint versions in 8.x. After the Elastic Stack upgrade, you can decide whether to upgrade Elastic Agents to 8.x, which is recommended to ensure you get the latest features.

You do not need to shut down your Elastic Agents or endpoints to upgrade the Elastic Stack.

Re-enable disabled rules

edit

Any rules that are active when you upgrade from 7.17 to 8.0.1 or newer are automatically disabled, and a tag named auto_disabled_8.0 is added to those rules for tracking purposes. Once the upgrade is complete, you can filter rules by the newly added tag, then use bulk actions to re-enable them:

  1. Go to the Rules page (Detect → Rules).
  2. From the Tags dropdown, search for auto_disabled_8.0.
  3. Click Select all x rules, or individually select the rules you want to re-enable.
  4. Click Bulk actions → Enable to re-enable the rules.

Alternatively, you can use the Bulk rule actions API to re-enable rules.

Full Disk Access (FDA) approval for Elastic Endpoint

edit

When you manually install Elastic Endpoint, you must approve a system extension, kernel extension, and enable Full Disk Access (FDA). There is a new FDA requirement in 8.x. Refer to Elastic Endpoint requirements to review the required permissions.

Requirements to display Data views in the Elastic Security app

edit

To make the Data view option appear in an environment with legacy alerts, a user with elevated role privileges must visit the Elastic Security app, open a page that displays alert data (such as the Overview page), then refresh the page. The user’s role privileges must allow them to enable the detections feature in a Kibana space. Refer to Enable and access detections for more information.

If new alerts are generated in an upgraded environment without legacy alerts, refreshing any page with alert data in Elastic Security will make the Data view option appear in the Elastic Security UI.

New alert schema

edit

The system index for detection alerts has been renamed from .siem-signals-<space-id> to .alerts-security.alerts-<space-id> and is now a hidden index. Therefore, the schema used for alert documents in Elastic Security has changed. Users that access documents in the .siem-signals indices using the Elastic Security API must modify their API queries and scripts to operate properly on the new 8.x alert documents. Refer to how to query alert indices and review the new Alert schema.

New privileges required to view alerts and preview rules

edit
  • To view alerts, users need manage, write, read, and view_index_metadata privileges to two new indices, .alerts-security.alerts and .internal.alerts-security.alerts. Existing users who are upgrading to 8.x can retain their privileges to the .siem-signals index.
  • To preview rules, users need read access to the new .preview.alerts-security.alerts index. Refer to Detections prerequisites and requirements for more information.

Updates to indictor match rules

edit

Changes to the indicator match rule’s default threat indicator path might require you to update existing rules, or create new ones, after upgrading to 8.x. Be mindful of the following:

  • If an indicator match rule’s default threat indicator path was not defined before the upgrade, it will default to threatintel.indicator after the upgrade. This allows the rule to continue using indicator data ingested by Filebeat version 7.x. If a custom value was defined before the upgrade, the value will not change.
  • If an existing indicator match rule was configured to use threat indicator indices generated from Filebeat version 7.x, updating the default threat indicator path to threat.indicator after you upgrade to Elastic Stack version 8.x and Elastic Agent or Filebeat version 8.x configures the rule to use threat indicator indices generated by those later versions.
  • You must create separate rules to query threat intelligence indices created by Filebeat version 7.x and version 8.x because each version requires a different default threat indicator path value. Review the recommendations for querying alert indices.