Fleet and Elastic Agent 8.6.1

edit

Review important information about the Fleet and Elastic Agent 8.6.1 release.

Known issues

edit
Osquery live query results can take up to five minutes to show up in Kibana.

Details
A known issue in Elastic Agent may prevent live query results from being available in the Kibana UI even though the results have been successfully sent to Elasticsearch. For more information, refer to #2066.

Impact
Be aware that the live query results shown in Kibana may be delayed by up to 5 minutes.

Adding a Fleet Server integration to an agent results in panic if the agent was not bootstrapped with a Fleet Server.

Details

A panic occurs because the Elastic Agent does not have a fleet.server in the fleet.enc configuration file. When this happens, the agent fails with a message like:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x557b8eeafc1d]
goroutine 86 [running]:
github.com/elastic/elastic-agent/internal/pkg/agent/application.FleetServerComponentModifier.func1({0xc000652f00, 0xa, 0x10}, 0x557b8fa8eb92?)
...

For more information, refer to #2170.

Impact

To work around this problem, uninstall the Elastic Agent and install it again with Fleet Server enabled during the bootstrap process.

Changing the Elastic Agent log level can incorrectly disable inputs in supervised Beats.

Details

Data collection may be disabled when the Elastic Agent log level is changed. Avoid changing the Elastic Agent log level.

Upgrade to 8.6.2 to fix the problem. For more information, refer to #2232.

Fleet Server will crash when configured to use the Warning log level.

Details

Fleet Server will crash when configured to use the Warning log level. Do not use the Warning log level. Affected Fleet Server instances must be reinstalled to fix the problem.

Upgrade to 8.6.2 to fix the problem. For more information, refer to #2328.

The initial release of Elastic Agent 8.6.1 for MacOS contained an unsigned elastic-agent executable. On MacOS Ventura, upgrading from 8.6.1 to any other version will fail and can disable Elastic Defend protections.

Details
The initial release of Elastic Agent version 8.6.1 for MacOS contained an unsigned elastic-agent executable and a correctly signed endpoint-security executable. The endpoint-security executable implements the endpoint protection functionality of the Elastic Defend integration.

New functionality in MacOS Gatekeeper in MacOS Ventura prevents the unsigned elastic-agent executable from modifying the installation of the signed endpoint-security executable causing upgrades from affected 8.6.1 versions to fail. The failed upgrade can leave Elastic Agent in an unhealthy state with the endpoint-security executable disabled. Note that MacOS Gatekeeper implements a signature cache, such that the upgrade is only likely to fail on MacOS Ventura machines that have been rebooted since the first upgrade to version 8.6.1.

As of February 27th 2023 the Elastic Agent 8.6.1 artifacts for MacOS have been updated with a correctly signed elastic-agent executable. To verify that the signature of the elastic-agent executable is correct, run the command below and ensure that Elasticsearch, Inc appears in the Authority field.

tar xvfz elastic-agent-8.6.1-darwin-aarch64.tar.gz
cd elastic-agent-8.6.1-darwin-aarch64
codesign -dvvvv ./elastic-agent
...
Signature size=9068
Authority=Developer ID Application: Elasticsearch, Inc (2BT3HPN62Z)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=Feb 24, 2023 at 4:33:02 AM
...

Impact
Any Elastic Agent deployed to MacOS Ventura that was upgraded to version 8.6.1 prior to February 27th 2023 must be reinstalled using a version with correctly signed executables. Upgrades to any other version will fail and lead to broken functionality, including disabling the protections from Elastic Defend.

The specific steps to follow to correct this problem are:

  1. Download a version of Elastic Agent with correctly signed executables.
  2. Unenroll the affected agents, either from the command line or the Fleet UI. A new agent ID will be generated when reinstalling.
  3. Run the elastic-agent uninstall command to remove the incorrectly signed version of Elastic Agent.
  4. From the directory containing the new, correctly signed Elastic Agent artifacts run the elastic-agent install command. The agent may be reenrolled at install time or separately with the elastic-agent enroll command.
Installing Elastic Agent on MacOS Ventura may fail if Full Disk Access has not been granted to the application used for installation.

Details
This issue occurs on MacOS Ventura when Full Disk Access is not granted to the application that runs the installation command. This could be either a Terminal or any custom package that a user has built to distribute Elastic Agent.

For more information, refer to #2103.

Impact
Elastic Agent will fail to install and produce "Error: failed to fix permissions: chown elastic-agent.app: operation not permitted" message. Ensure that the application used to install Elastic Agent (for example, the Terminal or custom package) has Full Disk Access before running sudo ./elastic-agent install.

Elastic Agent upgrades scheduled for a future time do not run.

Details
A known issue in Elastic Agent may prevent upgrades scheduled to execute at a later time from running. For more information refer to #2343.

Impact
Kibana may show an agent as being stuck with the Updating status. If the scheduled start time has passed, you may force the agent to run by sending it any action (excluding an upgrade action), such as a change to the policy or the log level.

Fleet ignores custom server.* attributes provided through integration settings.

Details
Fleet will ignore any custom server.* attributes provided through the custom configurations yaml block of the intgration. For more information refer to #2303.

Impact
Custom yaml settings are silently ignored by Fleet. Settings with input blocks, such as Max agents are still effective.

Bug fixes

edit
Fleet
  • Fix missing policy ID in installation URL for cloud integrations #149243
  • Fix package installation APIs to install packages without a version #149193
  • Fix issue where the latest GA version could not be installed if there was a newer prerelease version in the registry #149133 #149104
Fleet Server
  • Update the .fleet-agent index when acknowledging policy changes when Logstash is the configured output. Fixes agents always showing as Updating when using the Logstash output #2119
Elastic Agent
  • Fix issue where Beats started by Elastic Agent may fail with an output unit has no config error #2138 #2086
  • Restore the ability to set custom HTTP headers at enrollment time. Fixes traffic filters in Integrations Server cloud deployments #2158 #32993
  • Make it easier to filter agent logs from the combined agent log file #2044 #1810