Fleet and Elastic Agent 8.12.1
editFleet and Elastic Agent 8.12.1
editReview important information about Fleet Server and Elastic Agent for the 8.12.1 release.
Breaking changes
editBreaking changes can prevent your application from optimal operation and performance. Before you upgrade, review the breaking changes, then mitigate the impact to your application.
Naming collisions with Fleet custom ingest pipelines
Summary
If you were relying on an ingest pipeline of the form ${type}-${integration}@custom
introduced in version 8.12.0 (for example, traces-apm@custom
, logs-nginx@custom
, or metrics-system@custom
) you need to update your pipeline’s name to include an .integration
suffix (for example, logs-nginx.integration@custom
) to preserve your expected ingestion behavior.
Details
In version 8.12.0, Fleet added new custom ingest pipeline names for adding custom processing to integration data streams. These pipeline names used patterns as follows:
-
global@custom
-
${type}@custom
(for exampletraces@custom
) -
${type}-${integration}@custom
(for exampletraces-apm@custom
) -
${type}-${integration}-${dataset}@custom
pre-existing (for exampletraces-apm.rum@custom
)
However, it was discovered in #175254 that the ${type-integration}@custom
pattern can collide in cases where the integration
name is also a dataset name. The clearest case of these collisions was in the APM integration’s data streams, for example:
-
traces-apm
-
traces-apm.rum
-
traces-apm.sampled
Because traces-apm
is a legitimate data stream defined by the APM integration (see the relevant manifest.yml file), it incurred a collision of these custom pipeline names on version 8.12.0. For example:
// traces-apm { "pipeline": { "name": "traces-apm@custom", // <--- "ignore_missing_pipeline": true } }
// traces-apm.rum { "pipeline": { "name": "traces-apm@custom", // <--- "ignore_missing_pipeline": true } }, { "pipeline": { "name": "traces-apm.rum@custom", "ignore_missing_pipeline": true } }
Prior to version 8.12.0, the traces-apm@custom
custom pipeline name was already supported. So, if you had already defined and were using the supported traces-apm@custom
pipeline, and then upgraded to 8.12.0, you would observe that documents ingested to traces-apm.rum
and traces-apm.sampled
would also be processed by your pre-existing traces-apm@custom
ingest pipeline. This could cause breakages and unexpected pipeline processing errors.
To correct this in version 8.12.1, Fleet now appends a suffix to the "integration level" custom ingest pipeline name. The new suffix prevents collisions between datasets and integration names moving forward. For example:
// traces-apm { "pipeline": { "name": "traces-apm.integration@custom", // <--- Integration level pipeline "ignore_missing_pipeline": true } }, { "pipeline": { "name": "traces-apm@custom", // <--- Dataset level pipeline "ignore_missing_pipeline": true } }
// traces-apm.rum { "pipeline": { "name": "traces-apm.integration@custom", // <--- Integration level pipeline "ignore_missing_pipeline": true } }, { "pipeline": { "name": "traces-apm.rum@custom", // <--- Dataset level pipeline "ignore_missing_pipeline": true } }
So, if you are relying on an integration level custom ingest pipeline introduced in version 8.12.0, you need to update its name to include the new .integration
suffix to preserve your existing ingestion behavior.
Refer to the Ingest pipelines documentation for details and examples.
Known issues
editAgents upgraded to 8.12.0 are stuck in a non-upgradeable state
Details
An issue discovered in Fleet Server prevents Elastic Agents that have been upgraded to version 8.12.0 from being upgraded again, using the Fleet UI, to version 8.12.1 or higher.
Impact
As a workaround, we recommend you to use the Kibana Fleet API to update any documents in which upgrade_details
is either null
or not defined. Note that these steps must be run as a superuser.
POST _security/role/fleet_superuser { "indices": [ { "names": [".fleet*",".kibana*"], "privileges": ["all"], "allow_restricted_indices": true } ] }
POST _security/user/fleet_superuser { "password": "password", "roles": ["superuser", "fleet_superuser"] }
curl -sk -XPOST --user fleet_superuser:password -H 'content-type:application/json' \ -H'x-elastic-product-origin:fleet' \ http://localhost:9200/.fleet-agents/_update_by_query \ -d '{ "script": { "source": "ctx._source.remove(\"upgrade_details\")", "lang": "painless" }, "query": { "bool": { "must_not": { "exists": { "field": "upgrade_details" } } } } }'
DELETE _security/user/fleet_superuser DELETE _security/role/fleet_superuser
After running these API requests, wait at least 10 minutes, and then the agents should be upgradeable again.
Bug fixes
edit- Fleet
-
- Fix the display of category label on the Integration overview page. (#176141)
- Fix conflicting dynamic template mappings for intermediate objects. (#175970)
- Fix reserved keys for Elasticsearch output YAML box. (#175901)
- Prevent deletion of agent policies with inactive agents from UI. (#175815)
- Fix incorrect count of agents in bulk actions. (#175318)
- Fix a custom integrations not displaying on the Installed integrations page. (#174804)
- Elastic Agent