Elastic Cloud Enterprise 2.13.0

New deployment template design. The deployment template view and edit pages have been redesigned. The new design adds the hot/warm/cold/frozen tiers, and automatically includes the correct data to the template for those tiers to work correctly.

Official Elastic diagnostics tool now supported. In an effort to streamline how we support you, Elastic Cloud Enterprise now integrates the Elasticsearch and Kibana diagnostics tooling that comes with the stack.

The following changes are included in this release.

Allocator to support service tokens in Elasticsearch configuration. Allocator now provides service tokens to template renderer that renders the stack packs template files of stateless apps (APM, Kibana, etc). Currently only one known token is supported for APM (Fleet-server). It is available by key fleetServerServiceToken.

Constructor to generate service tokens for stateless clusters. Constructor to support generating service tokens for stateless clusters and support for APM 7.14 service tokens.

Elasticsearch error code reporting. Elasticsearch startup failures will be categorized and reported to constructor based on the logged exit code

Only use admin-proxy for ECE platform admins. The Elasticsearch feature Operator Privileges is now enabled for the API console on ECE. Only users with the Platform Admin role will be able to execute requests that are marked as operators-only.

Autoscaling. Adds the ability to enable autoscaling by default on deployment templates, and adds dropdowns to configure autoscaling min and max values for data tiers and machine learning.

Migrate to node_roles when enabling warm tier. Enabling the warm data tier will now automatically migrate a deployment configuration from using node_type to using node_roles. To learn more about the new node_roles field, check Changes to index allocation and API.

Add maxZones field to InstanceConfig models from query to AZs of allocators. The InstanceConfiguration and InstanceConfigurationInfo models present in our rest APIs now have an extra, optional field (read-only). This field is max_zones and it defines the maximum number of zones in which a given instance configuration has allocators.

Split up internal and external services model, add typed TLS service in API spec. Adds the information of the upgraded service in the successful responses to POST /platform/configuration/security/tls/{service_name} requests. - Adds typed Swagger specification on the list of API managed services, the UpdatedTlsChain object.

Enable configuration settings for Workplace Search synonym limits and remote query timeout. The following settings are now available in Elastic Cloud Enterprise:

Automatically show a traffic filters flyout for URL parameters. Opening a /traffic-filters?filter_type=ip&provider=azure&region=[FROM AZURE] URL, now shows a traffic filter page with the create flyout present and IP address preselected. Opening a /traffic-filters?filter_type=endpoint&provider=azure&region=[FROM AZURE]&resource_name=[FROM AZURE]&resource_id=[FROM AZURE] URL shows a traffic filter page with the Create traffic filter flyout present and the Private link endpoint option preselected. The Region, Resource name, and Resource ID fields are pre-populated.

UI for capturing thread dumps. Adds button to UI to capture thread dumps.

Add support for OOM heap dumps to Enterprise Search. OOM heap dump support has been extended to Enterprise Search instances. For Enterprise Search instances you can capture a heap dump on an Out of Memory event and it will be accessible through the Elastic Cloud Enterprise UI and API.

Limit compatible versions for CCS/CCR, starting with 8.0. This release of Elastic Cloud Enterprise updates the rules for which versions are compatible to be used together for cross-cluster search and cross-cluster replication.

Add Kibana diagnostics. Adds the POST /deployments/{deployment_id}/kibana/{ref_id}/diagnostics/_capture endpoint to retrieve Kibana diagnostics.

Track machine learning usage in all environments. Track the machine learning feature via the GET /api/v1/saas/billing/usage/features API in all environments.

Reduce docker health-related plan failures. When docker is not healthy, allocators are not able to start new instances, which will cause failed plans. This change makes sure that allocators where docker is not healthy are avoided when allocating new instances.

New deployment template design. Redesign of deployment template view and edit pages. Adds hot/warm/cold/frozen tiers, and automatically adds the correct data to the template for those tiers to work correctly. Shows all sliders by default, so the user does not have to add them. Fixes some bugs as well.

Update Backend Response Time to include a list of attempted instances. Adds a list of backend_response_time for all the instances that the proxy attempted. This helps to see how many instances the proxy tries to connect to and the response time from each of those instances.

Don’t run SetQuorumSize for 7.x clusters. Plans for version 7.x and later clusters will no longer execute the step "Setting Elasticsearch cluster quorum size".

Internal certification warnings. Show warnings on Platform pages when internal CA TLS certificates are close to or fully expired.

Validate that every resource has at least one topology element. Additional validation to make sure that all resources in a deployment template have at least one topology element. This was already required before, but is now validated when creating or updating deployment templates.

When upgrading to 7.14, default to migrating to node_roles. A deployment being upgraded to version 7.14 or higher defaults to migrating from the deprecated node_type to node_roles.

Thread dumps in Enterprise Search. Adds support for taking a thread dump from Enterprise Search instances.

Add Elasticsearch diagnostics. Adds the POST /deployments/{deployment_id}/elasticsearch/{ref_id}/diagnostics/_capture endpoint to retrieve cluster diagnostics.

Changing default fallback CAPACITY. Changed the default for CAPACITY so that the remainder maxes out at 8GB and to align with discrete GB as instances also require discrete GB to be assigned.

Set nameid_format in SAML config for SSO. Elastic Cloud Enterprise now sets nameid_format explicitly when configuring SAML realms for SSO.

Make the document-level security and field-level security features available in all environments. The DLS and FLS features are now accessible through the GET /api/v1/saas/billing/{organization_id}/features API in all environments.

Add workplace_search.content_source.salesforce.enable_cases setting. Added configurable settings for Salesforce content source in Workplace Search (workplace_search.content_source.salesforce.enable_cases) that allow you to enable or disable synchronization of Case Salesforce objects.

Add help text to region field. When creating a traffic filter a region field must be set, however there can be a few different concepts of region at this time. This PR adds some clarifying help text to the region field shown on the Create traffic filter UI.

EOL warnings in stack versions. Displays an End of Life warning on the Elastic Cloud Enterprise UI where the stack version and deployment lists are displayed, and the individual deployment pages, for Elastic Stack packs and deployments using versions 5.x and 2.x.

Informational warning about default certificates. Adds a warning when the initial installation certificates are in use, rather than user-managed certificates (e.g proxy or ui).

Don’t overwrite use_for_peer_recovery if already set in the repository settings. Prevent overwriting use_for_peer_recovery when it’s defined by ECE users at the repository level. This setting can be explicitly disabled if accessing snapshot repositories incurs an additional cost.

Dedicated failure when running out of capacity. Adds a dedicated error class for cases when the platform does not have enough capacity to allocate an instance.

Traffic filters UI: set initial state of region selector to empty, to match component internal state. Fixed a bug that prevented users from creating traffic filters when the default cloud provider and region were selected.

Show failed status and error in heap dump UI. Fixes a UI error when a heap dump capture fails.

Make sure every cluster data contains a self-reference to the same cluster. Fixes the cluster metadata so internal requests to Kibana (for example, the Kibana proxy endpoints) work when using traffic filters.

Remove the APM rate_limit max values. Removes the maximum value limits of 100, 100 and 3000 for the APM server settings:

Prevent crashes on pages translated with Google Translate. Fixes crashes in certain interactions of the login, reset password, and billing pages when they are translated using Google Translate.

Fixed a bug that applied the default traffic filters when a deployment is edited. When existing deployments were edited, even for no-op plans, the default traffic filters were being applied.

Add index level privileges for data streams. Adds create and write privileges to the cloud-internal-apm-server role for data streams.

Do not meter hidden deployments. This PR updates the CTS feature-usage task to check if the cluster instance it is about to meter usage for belongs to a hidden deployment. If so, the task does not perform any metering of that instance.

Close terminate modal when clicking terminate. Fixes a small bug where the Terminate Deployment modal did not close after clicking Terminate.

Fixed SAML principal mapping docs and logs. ECE SAML Authentication Provider requires stricter validation on the types of fields that can be used to map to the principal SAML attribute, in order to support SSO from the ECE admin console to Kibana. For example, the identifier cannot contain commas or slashes. Rather than failing to log in users mapped with these invalid identifiers, they can now log into ECE correctly and only be blocked from using the built-in SSO into Kibana, with an error logged to the adminconsole log file explaining the problem. The documentation has also been updated to better warn about this caveat.

Extend the versions for which we disable concurrent snapshots to 7.14.0. Elasticsearch versions prior to 7.14.1 suffer from a rare but data-loss-causing bug that can silently and prematurely delete data from a snapshot repository. As a workaround, ECE disables concurrent snapshots in all 7.9.0 - 7.14.0 deployments by setting snapshot.max_concurrent_operations: 1 in elasticsearch.yml. The change is applied when a deployment is created or modified. When a deployment is upgraded to 7.14.1 or higher, the workaround is removed.

Stop setting allocation excludes in SetQuorumSize. The set-quorum-size step no longer adds unnecessary allocation excludes for 6.x Elasticsearch clusters. However, they are still added for version 6.1.0-to-6.1.2 clusters in order to work around a bug in those versions of Elasticsearch.

Use -f when copying log delivery start scripts. Fixed a rare issue where Elasticsearch nodes failed to restart with Text file busy when the log delivery feature was enabled on some ECE configurations.

Respect extendedMaintenance during rolling step. Fixed a bug which caused plans to disable maintenance mode too early when applying rolling changes to nodes during a plan which uses extended maintenance.

Improve withValidAdminConsoleClient behavior. The ECE upgrader had a bug where the --api-base-url parameter might be a valid URL but is still ignored due to the Adminconsole API being in an unhealthy state. This has been fixed so that any URL that is able to connect to the Adminconsole and receive any HTTP response is considered a valid URL.

Remove curation and ILM configuration from Create and Edit Deployment. The UI for configuring index curation has been removed. For any older deployments that require index curation changes, the API remains available.

Whitelist the vis_type_vega.enableExternalUrls. In Elastic Stack version 8.0.0, the vega.enableExternalUrls YAML setting will be removed (elastic/kibana#112643). Use the vis_type_vega.enableExternalUrls setting instead.

Supports timelion.ui.enabled until v7.16. Removes support for the timelion.ui.enabled kibana.yml setting due to the removal of the Timelion app in the Elastic Stack version 7.16.

Adds a max version in the vis_type_table.legacyVisEnabled setting. The vis_type_table.legacyVisEnabled setting is supported until Elastic Stack version 7.16.x.

Depreciation of cross-cluster search deployments. The dedicated cross-cluster search deployment type is deprecated in preparation for its removal in Elastic Cloud Enterprise 3.0. It is no longer listed when creating new deployments through the UI.

kibana-defaultAppId unsupported in version 8.0'. The kibana.defaultAppId setting has been added to the blocklist for version 8.0.0, as the property is no longer supported in version 8.

The following set endpoints will be removed from the Elastic Cloud Enterprise RESTful API in version 3.0. See [link to come] for options on how to migrate to a different endpoint before upgrading ECE.

Add workplace_search.content_source.salesforce.enable_cases setting. Adds configurable settings for Salesforce content source in Workplace Search (workplace_search.content_source.salesforce.enable_cases) that allows to enable or disable synchronization of Case Salesforce objects.

Fixing missing and incorrect doc links for ECE auth provider pages. The ECE documentation links for all Authentication Providers have been updated to point to the correct provider documentation sections, rather than the more general RBAC documentation.

Traffic Filter API Example. We’ve added several examples to help you manage traffic filters in Elastic Cloud Enterprise using the API.