Fleet and Elastic Agent 8.10.0

Details
The mechanism that Fleet uses to generate diagnostic bundles has been updated. To collect Elastic Agent diagnostics, Fleet Server needs to be at version 8.10.0 or higher.

Impact
If you need to access a diagnostic bundle for an agent, ensure that Fleet Server is at the required version.

Details

Starting from version 8.9.0, when Elastic Agent tries to perform an upgrade, it first verifies the binary signature with the key bundled in the agent. This process has a backup mechanism that will use the key coming from https://artifacts.elastic.co/GPG-KEY-elastic-agent instead of the one it already has.

In an air-gapped environment, the agent won’t be able to download the remote key and therefore cannot be upgraded.

Impact

For the upgrade to succeed, the agent needs to download the remote key from a server accessible from the air-gapped environment. Two workarounds are available.

Option 1

If an HTTP proxy is available to be used by the Elastic Agents in your Fleet, add the proxy settings using environment variables as explained in Proxy Server connectivity using default host variables. Please note that you need to enable HTTP Proxy usage for artifacts.elastic.co to bypass this problem, so you can craft the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment variables to be used exclusively for it.

Option 2

As the upgrade URL is not customizable, we have to "trick" the system by pointing https://artifacts.elastic.co/ to another host that will have the file.

The following examples require a server in your air-gapped environment that will expose the key you will have downloaded from https://artifacts.elastic.co/GPG-KEY-elastic-agent`.

Example 1: Manual

Edit the Elastic Agent server hosts file to add the following content:

<YOUR_HOST_IP> artifacts.elastic.co

The Linux hosts file path is /etc/hosts.

Windows hosts file path is C:\Windows\System32\drivers\etc\hosts.

Example 2: Puppet

host { 'elastic-artifacts':
  ensure       => 'present'
  comment      => 'Workaround for PGP check'
  ip           => '<YOUR_HOST_IP>'
}

Example 3: Ansible

- name  : 'elastic-artifacts'
  hosts : 'all'
  become: 'yes'

  tasks:
    - name: 'Add entry to /etc/hosts'
      lineinfile:
        path: '/etc/hosts'
        line: '<YOUR_HOST_IP> artifacts.elastic.co'

Details

A KQL query in a Fleet search field now returns a 400 error when the query is not valid.

Previously, the search fields would accept any type of query, but with the merge of #161064 any type of KQL sent to Fleet needs to have a valid field name, otherwise it returns an error.

Cause

Entering an invalid KQL query on one of the Fleet KQL search fields or through the API produces the error.

Affected search fields in the Fleet UI:

Affected endpoints in the Kibana Fleet APIs (these are the endpoints that accept the parameter ListWithKuery):

Impact

To avoid getting the 400 error, the queries should be valid.

For instance, entering the query 8.10.0 results in an error. The correct query should be: local_metadata.agent.version="8.10.0".

As another example, when viewing the Agents tab in Fleet, typing a hostname such as a0c8c88ef2f5 in the search field results in an error. The correct query should have the correct field name, taken from among the allowed ones, for example local_metadata.host.hostname: a0c8c88ef2f5.

The list of available field names is visible by clicking on any of the search fields.