Elastic Cloud Enterprise 3.4.0

We have identified an issue with the proxy and route-server component that can lead to slow requests, timeouts, and reduced throughput. For now, we recommend skipping this version until we have a bug fix released. If you have already upgraded and are experiencing this issue, please reach out to Elastic Support for a workaround. 

Making it easier to search and replicate data across multiple deployments. A new user interface enables setting up remote connections to other clusters. These clusters can be of mixed deployment type, both on premises and Elastic Cloud, giving full flexibility on where to store and use your data.

Deprecating the Elastic Amazon Machine Images (AMIs). You can no longer install Elastic Cloud Enterprise with Elastic Amazon Machine Images (AMIs). We recommend that you install ECE with Ansible.

The following changes are included in this release.

Show CA certificate metadata and download button for deployment. Added a new feature to download a deployment’s CA certificate. This enables users to connect their self-managed and ECE deployments to Elasticsearch Service.

Adminconsole: enable self-monitoring on deployment creation. Extension to the create-deployment API call: the observability destination for logging and metrics can now be set to self. This enables self-monitoring on deployment creation.

Add name to account trust. Added a name property to Account Trust Relationship.

Add also_trusted_by to certificate metadata in direct trusts. Certificate metadata in DirectTrustRelationships now lists other deployments that also trust the certificate.

Allow csp.disableUnsafeEval configuration in kibana.yml. Starting in Kibana 8.3, it’s possible to set the experimental csp.disableUnsafeEval configuration option. Set this to true to remove the unsafe-eval source expression from the script-src Content Security Policy (CSP) directive. The default value is false, which is identical to the original Kibana behavior.

By enabling csp.disableUnsafeEval, Kibana will use a custom version of the Handlebars template library which doesn’t support inline partials. Handlebars is used in various locations in the Kibana frontend where custom templates can be supplied by the user when for instance setting up a visualization. If you experience any issues rendering Handlebars templates after turning on csp.disableUnsafeEval, or if you rely on inline partials, please revert this setting to false and open an issue in the Kibana GitHub repository.

Virtualize the code block to improve performance. API Console output is now virtualized (only visible parts will be loaded at a given time to improve performance).

Enable vacates with closed indices for Elasticsearch versions prior to 7.2. Added the option to vacate nodes that are part of clusters containing closed indices, for Elasticsearch versions prior to 7.2.0.

Added version mismatch UI. If an upgrade fails, users will now be directed to complete that upgrade before doing anything else with the deployment, as other actions would likely have failed.

Set the default HTTPS port to 443 for production environments. Set the default port for Elasticsearch endpoints displayed in the ESS Cloud UI/API to 443 (the default HTTPS port). Port 9243 will continue to route as expected.

Log request bodies to the Adminconsole API log. Log requests payloads as part of the requests entries in the Adminconsole log. These are redacted using the API model privacy tags. Additionally, all proxied requests (deployments/<id>/type/resource/<proxied_route>) are considered secrets due to being prone to leak secrets and sensitive pieces of information.

Renamed upstream to backend. Renamed upstream_proto in the proxy logs to backend_proto to maintain consistency with other proxy log fields relating to the connection between the proxy and the stack components. Renamed upstream_dialer to backend_dialer for same reason.

Changed the verbiage for frozen nodes and the cache storage. Changed the wording on frozen instances when displaying how much of the blob storage is full, based on the disk storage.

Added description, size and last modified to the table for extensions. Added 3 new fields to the table showing your extensions. The table now lists the description field, the size of the extension, and the last time it was modified.

New migrations.discardCorruptObjects Kibana setting. When upgrading the Elastic Stack, Kibana runs a migration process to ensure system indices are up-to-date with the newer version. Sometimes, migrations might fail due to data inconsistencies. For instance, if corrupt objects exist or transform errors occur during a migration, this will cause the migration to fail.

This version adds support for the new migrations.discardCorruptObjects flag. This flag tells Kibana to discard corrupt saved objects that can be found during a migration, as well as transform errors, logging a warning message and carrying on with the migration process.

Note that this flag must be set to the target version, that is, the version to which the Elastic Stack is being upgraded. For instance, if you are upgrading to 8.4.0, you must set migrations.discardCorruptObjects: "8.4.0" in order for the flag to be taken into account. This is a safeguard that prevents Kibana to systematically ignore / discard unknown objects in future migrations, in case users forget to disable the flag after a migration.

New migrations.discardUnknownObjects Kibana setting. When upgrading the Elastic Stack, Kibana runs a migration process to ensure system indices are up-to-date with the newer version. Sometimes, migrations might fail due to data inconsistencies. For instance, if saved objects exist which have unknown types, this will cause the migration to fail.

This version adds support for the new migrations.discardUnknownObjects flag. This flag tells Kibana to discard unknown saved objects that can be found during a migration, logging a warning message and carrying on with the migration process.

Note that this flag must be set to the target version, that is, the version to which the Elastic Stack is being upgraded. For instance, if you are upgrading to 8.4.0, you must set migrations.discardUnknownObjects: "8.4.0" in order for the flag to be taken into account. This is a safeguard that prevents Kibana to systematically ignore / discard unknown objects in future migrations, in case users forget to disable the flag after a migration.

Allow proxy to establish TLSv1.3 connections with willing clients. Enable TLS v1.3 on the proxies. Note, establishing TLS v1.2 connections is still possible and not going away for now; this is an extra option for clients that support TLS v1.3.

Added a Deployment Id section to the new header. Added a more prominent and copyable location for the Deployment Id in the header of a deployment.

Allow http.connection_pool_ttl for OIDC realms. Allow user override of the new http.connection_pool_ttl setting for OIDC realms.

Move settings to plan level. The enabled_built_in_plugins, user_plugins, and user_bundles configuration entries will be moved from the cluster_topology level to the plan level when a plan change occurs through the UI.

Override clear upgrade version. Users can now clear administrator set user setting overrides without having to file a support ticket during the version upgrade process.

Bump Beats version to 7.17.5. Upgraded Filebeat and Metricbeat, used to ingest logs and metrics in ECE, to 7.17.5.

Allow users to clear override settings on the Edit page. Users are now able to clear administrator set user setting overrides without needing to file a support ticket. Go to the Edit page of your deployment and if an override is set, select the Clear overrides button.

Expose deployment.autoscaling_enabled fields in the API models. Introduce autoscaling_enabled at the deployment level. This will be consistent with the autoscaling_enabled field on the Elasticsearch resource. In the future, this value will also be used to enable autoscaling on other resources within a deployment.

Log Initial Data Step added to all flows. This change adds a new step named Log Initial Data to the UI, which displays cluster_id, plan_id and constructor for all flows. An earlier change displayed this info, but this step was only applicable to the modification/edit flow. With this change, cluster_id, plan_id and constructor will be displayed for all flows.

Instance capacity overrides are enforced for Elasticsearch heap when reset. Fixes the problem when instance overrides are reset through a next plan change, but the Elasticsearch heap settings remain unchanged.

Allocators API: Obey parameters in allocator listing requests when no filter are passed. Allocator listings now obey size, order and paging parameters, even when search criteria are not passed. This protects against huge responses when sizing limits are ignored.

Add check for haproxy configs in Docker health check. Add verification of haproxy config files to Docker /healthcheck. This should alert us if an invalid config is written to the disk, but an existing haproxy continues to run.

TermQuery.value is not an object type. Updates the swagger type of TermQuery.value to String. This should accept a combination of string | number | boolean. However, a framework limitation precludes supporting combination types in our swagger definition.

Don’t send payload on console API if strings are empty. Fixes a bug where content-type was being sent when no body existed.

Revert "Cluster ids are always in a should clause, regionId is a must". Fixes an issue where trusting more than one specific deployment would lead to those deployments being listed as (deleted) in the deployment trust UI.

Enterprise Search template rendering. The Enterprise Search section of templates in ECE should now render correctly in the UI.

Upgrade issues when deleting Elastic Stack versions. Fixes a bug where ECE users delete an Elastic Stack version, and any deployment on that version cannot be upgraded through the UI.

Re-enable monitoring cleaner service. Fixes monitoring indices not being cleaned up regularly.

When using the logging and monitoring feature, monitoring data is supposed to be removed after a retention period (default: 3 days). A bug led to the data being stored indefinitely. The monitoring indices are now cleaned up correctly after the configured retention period.

Don’t rebuild the topology element when configuring autoscaling_min. Fixes an issue preventing users from updating the maximum size of ML tiers.

Added fields to APM allow list. Adds the following settings to the APM allow list:

LDAP/AD bind credentials validation. With Elasticsearch LDAP and Active Directory security realms, users will no longer be able to set the bind DN field without setting a secure bind password. To make sure ECE can successfully upgrade its security cluster, ECE now validates that all existing LDAP and Active Directory Authentication Providers do not have a Bind DN field without a matching password, and will block the upgrade if they do. The ECE LDAP and Active Directory configuration APIs will now return a 400 Bad Request Error if users try to set the bind_dn field without a bind_password.

Remove legacy cipher suite from aws eu-west-1. Updated aws eu-west-1 region cipher suites to intermediate ciphers + Windows 11 client compatible cipher suites.

Remove Adminconsole from haproxy backend. Removed inbound port 12344 that used to allow internal components of ECE to connect to Admin API, aligned with Networking prerequisites.

Set maximum version for xpack.apm.maxServiceEnvironments. xpack.apm.maxServiceEnvironments has been deprecated in 8.0.0. The setting has been replaced by maxSuggestions which can be configured in Kibana’s Advanced Settings.