Fleet and Elastic Agent 8.13.0

edit

Review important information about Fleet Server and Elastic Agent for the 8.13.0 release.

Security updates

edit
Elastic Agent
  • Update Go version to 1.21.8. #4221

Breaking changes

edit

Breaking changes can prevent your application from optimal operation and performance. Before you upgrade, review the breaking changes, then mitigate the impact to your application.

Removes conditional topics for Kafka outputs

Details
The Kafka output no longer supports conditional topics while the final syntax is evaluated ahead of Kafka output GA. For more information, refer to (#176879).

Most Fleet installed integrations are now read-only and labelled with a Managed tag in the Kibana UI

Details
Integration content installed by Fleet is no longer editable. This content is tagged with Managed in the Kibana UI, and is Elastic managed. This content cannot be edited or deleted, however managed visualizations, dashboards, and saved searches can be cloned. The clones can be customized. When cloning a dashboard the cloned panels become entirely independent copies that are unlinked from the original configurations and dependencies. Managed content relating to specific visualization editors such as Lens, TSVB, and Maps, the clones retain the original reference configurations. The same applies to editing any saved searches in a managed visualization. For more information, refer to (#172393).

The behavior of queue.mem.flush.min_events has been simplified.

Details
The behavior of queue.mem.flush.min_events has been simplified. It now serves as a simple maximum on the size of all event batches. There are no longer performance implications in its relationship to bulk_max_size.

For more information, refer to (#37795).

Notable changes

edit

The following are notable, non-breaking updates to be aware of:

  • Changes to features that are in Technical Preview.
  • Changes to log formats.
  • Changes to non-public APIs.
  • Behaviour changes that repair critical bugs.

    Fleet
  • Adds reference to ecs@mappings for each index template (#174855).

Known issues

edit
Beats MSI binaries do not support directories with a trailing slash

Details

Due to changes introduced to support customizing an MSI install folder (see #209), Beats MSI binaries, which currently are in beta, will not properly handle directories that end in a slash. This defect may affect many deployments using the Beats MSI binaries.

Impact

This issue has been resolved in version 8.14.0 and later releases. We recommend users of Beats MSI to upgrade to 8.14 when that release becomes available.

New features

edit

The 8.13.0 release added the following new and notable features.

Fleet
  • Adds support for the subobjects setting on the object type mapping (#171826).
Fleet Server
  • Add support for storing output secrets in a new secrets block. 3061 2966
  • Add support for the remote Elasticsearch output type in Fleet Server. 3051
  • Report the health state of remote Elasticsearch outputs to the logs-fleet_server.output_health-default data stream. 3127 3116
  • Add a policy_debounce_time configuration to add a forced delay to the policy index monitor when it successfully gathers new documents. 3234
Elastic Agent
  • Log a summary of each policy configuration change received from Fleet. #4050 #3406
  • Add the full version number to the installation directory name. #4193 #2579
  • Ignore Kubernetes node and namespace update events that do not change pod metadata. #4226 #37338
  • Add the new ETW input mapping to the Filebeat specification so that it’s available in Elastic Agent. #4037 #36915
  • Add the new WebSocket input mapping to the Filebeat specification so that it’s available in Elastic Agent. #4242 #37774
  • Create the .installed marker earlier on in the install process, allowing the use of elastic-agent uninstall to cleanup if the install fails. #4172 #4051
  • Add a postrm script to Elastic Agent DEB and RPM packages. #4334 #3784 #4267
  • Kubernetes secrets provider has been improved to update a Kubernetes secret when the secret value changes. #4371 #4168
  • Upgrade elastic-agent-system-metrics to version 0.9.2. #4383
  • Allow users to configure number of output workers (for outputs that support workers) with either worker or workers. 38257

Enhancements

edit
Fleet
  • Adds skipRateLimitCheck flag to the Upgrade API and Bulk_upgrade API (#176923).
  • Adds making datastream rollover lazy (#176565).
  • Stops creating the {type}-{datastet}@custom component template during package installation (#175469).
  • Adds the xpack.fleet.isAirGapped flag (#174214).
  • Add a warning when downloading the new version in an agent upgrade is failing (#173844).
  • Adds a message explaining why an agent is not upgradeable (#173253).
  • Makes logs-* and metrics-* data views available across all spaces (#172991).
  • Adds flag for pre-release to templates/inputs endpoint (#174471).
  • Adds concurrency control to Fleet data stream API handler (#174087).
  • Adds a handlebar helper to percent encode a given string (#173119).
Fleet Server
  • Relax version checks in snapshot builds to support automated testing during minor release updates. 3039 2960
  • Add top level keys for policy definition into Fleet Server OpenAPI specification. 3048
  • Define the action.data and ack event schemas. 3060
  • Add additional transaction labels with Elasticsearch error details to requests. 3124 3098
  • Calls with unauthorized API keys now return a 401 error. 3135 2861
  • Use the Shutdown method with a timeout to gracefully halt HTTP servers. 3165 2902
  • Replace the policy and action limiters with a unified checkin limiter. 3255 3254
  • Change the response code for Elasticsearch call failures to 503. 3235 2852
Elastic Agent
  • Move the control socket path to always be inside of the top level of the Elastic Agent installation directory. #3909 #3840
  • Add mTLS flags to Elastic Agent install and enroll commands to enable use of certificates for communication in on-prem proxy setups. #4007
  • Improve error handling by adding error descriptors to the inspect command and config methods. #4074
  • Add an agent.providers.initial_default configuration flag to disable providers by default. #4166 #4145
  • Add environment variable bindings so that Fleet Server and Elastic Agents started in container mode can specify mTLS variables. #4261

Bug fixes

edit
Fleet
  • Fixes a bug where secret values were not deleted on output type change (#178964).
  • Fixes formatting for some integrations on the overview page (#178937).
  • Fixes the name of Elasticsearch output workers configuration key (#178329).
  • Fixes clean up of the .fleet-policies entries when deleting an agent policy. (#178276).
  • Fixes only showing remote Elasticsearch output health status if later than last updated time (#177685).
  • Fixes status summary when showUpgradeable is selected (#177618).
  • Fixes issue of agent sometimes not getting inputs using a new agent policy with system integration (#177594).
  • Fixes the activity flyout keeping the scroll state on rerender (#177029).
  • Fixes inactive popover tour not resetting (#176929).
  • Fixes isPackageVersionOrLaterInstalled to check for installed package (#176532).
  • Removes pre-release exception for Synthetics package (#176249).
  • Fixes output validation when creating package policy (#175985).
  • Fixes allowing an agent to upgrade to a newer patch version than fleet-server (#175775).
  • Fixes asset creation during custom integration installation (#174869).
  • Fixes cascading agent policy’s namespace to package policies (#174776).
Fleet Server
  • Add missing Elastic-Api-Version and X-Request-Id headers to the Fleet Server OpenAPI specification. 3044
  • Replace all secret references in input objects. 3086 3083
  • Deprecate the redundant fleet.agent.logging.level attribute. 3195 3126
  • Add validation to make sure that status and message are present in the checkin API request body. 3233 2420
  • Fix a bug where agents were stuck in non-upgradeable state after an upgrade. 3264 3263
  • Fix chunked file delivery so that files are delivered in order. #3283
  • Fix a bug where the self monitor stops output health reporting if the output configuration is not acknowledged by agents. #3335 3334
Elastic Agent
  • Fix component control protocol to allow checkin to be chunked across multiple messages. Fixes errors related to the gRPC max message size being exceeded. #3884 #2460
  • Fix the creation of directories when unpacking tar.gz packages. #4100 #4093
  • Set a timeout of 1 minute for the FQDN lookup function. #4147
  • Increase timeout for file removal during Elastic Agent uninstall. #4310 #4164