Beats version 8.16.0

edit

Beats version 8.16.0

edit

View commits

Known issues

edit

Filebeat

The AWS S3 input polling mode is not working when the S3 bucket is not in the us-east-1 default region. This also impacts all AWS integrations and any custom AWS log integration which uses the aws-s3 input polling mode. When using Filebeat, please add a default_region configuration with the region of the S3 bucket. For example:
```
filebeat.inputs:
- type: aws-s3
  enabled: true
  credential_profile_name: elastic-observability
  default_region: us-east-2
  number_of_workers: 5
  bucket_arn: 'arn:aws:s3:::test1'
```

Metricbeat

Metrics can be lost when using Metricbeat due to the total fields limit of the Metricbeat index template. We recommend increasing the index.mapping.total_fields.limit setting of the Metricbeat index template to 12500 and perform a rollover of the Metricbeat data stream. If you’ve customized the name of the index associated to Metricbeat, apply the same change accordingly.

Breaking changes

edit

Affecting all Beats

Fix FQDN being lowercased when used as host.hostname. 39993
Beats won’t log start up information when running under the Elastic Agent. 40390
Filebeat now needs dup3, faccessat2, prctl and setrlimit syscalls to run the journald input. If this input is not being used, the syscalls are not needed. All Beats have those syscalls allowed now because the default seccomp policy is global to all Beats. 40061
Beats will rate limit the logs about errors when indexing events on Elasticsearch, logging a summary every 10s. The logs sent to the event log is unchanged. 40157

Filebeat

Filebeat, when running with Elastic-Agent, reports status for Filestream input. 40121
Added support for hyphens in extension keys in decode_cef Filebeat processor. 40427
Journald: removed configuration options include_matches.or, include_matches.and, backoff, max_backoff, cursor_seek_fallback. 40061
Journald: include_matches.match now behaves in the same way as matchers in journalctl. Users should carefully update their input configuration. 40061
Journald: seek and since behaviour have been simplified, if there is a cursor (state) seek and since are ignored and the cursor is used. 40061
Redis: Added replication role as a field to submitted slowlogs.
Added container.image.name to journald Filebeat input’s Docker-specific translated fields. 40450
Remove deprecated awscloudwatch field from Filebeat. 41089
The performance of ingesting SQS data with the S3 input has improved by up to 60x for queues with many small events. max_number_of_messages config for SQS mode is now ignored, as the new design no longer needs a manual cap on messages. Instead, use number_of_workers to scale ingestion rate in both S3 and SQS modes. The increased efficiency may increase network bandwidth consumption, which can be throttled by lowering number_of_workers. It may also increase number of events stored in memory, which can be throttled by lowering the configured size of the internal queue. 40699

Metricbeat

Add support for specifying a custom endpoint for GCP service clients. 40848 40918

Bugfixes

edit

Auditbeat

Request status from a separate socket to avoid data congestion. 41207

Filebeat

Fix crashes in the journald input. 40061
Fix long filepaths in diagnostics exceeding max path limits on Windows. 40909
Fix a bug in Salesforce input to only handle responses with 200 status code. 41015
Fixed failed job handling and removed false-positive error logs in the GCS input. 41142
Bump github.com/elastic/go-sfdc dependency used by x-pack/filebeat/input/salesforce. 41192
Journald input now can read events from all boots 41083 41244
Fix errors in SQS host resolution in the aws-s3 input when using custom (non-AWS) endpoints. 41504

Metricbeat

Add GCP instance_id resource label in ECS cloud fields. 40033 40062
Remove excessive info-level logs in cgroups setup. 40491
Fix http server helper SSL config. 39405

Added

edit

Filebeat

Implement Elastic Agent status and health reporting for Netflow Filebeat input. 40080
Add SSL and username support for Redis input, now the input includes support for Redis 6.0+. 40111
Add scaling up support for Netflow input. 37761 40122
Update CEL mito extensions to v1.15.0. 40294
Improve logging in Okta Entity Analytics provider. 40106 40347
Document winlog input. 40074 40462
Added retry logic to websocket connections in the streaming input. 40271 40601
Disable event normalization for netflow input. 40635
Allow attribute selection in the Active Directory entity analytics provider. 40482 40662
Improve error quality when CEL program does not correctly return an events array. 40580
Added support for Microsoft Entra ID RBAC authentication. 40434 40879
Add use_kubeadm config option for filebeat (both filbeat.input and autodiscovery) in order to toggle kubeadm-config api requests. 40301
Make HTTP library function inclusion non-conditional in CEL input. 40912
Add support for Crowdstrike streaming API to the streaming input. 40264 40838
Add support to CEL for reading host environment variables. 40762 40779
Add CSV decoder to awss3 input. 40896
Change request trace logging to include headers instead of complete request. 41072
Improved GCS input documentation. 41143
Add CSV decoding capacity to azureblobstorage input. 40978
Add CSV decoding capacity to gcs input. 40979
Add support to source AWS cloudwatch logs from linked accounts. 41188
Jounrald input now supports filtering by facilities. 41061
Add support to include AWS cloudwatch linked accounts when using log_group_name_prefix to define log group names. 41206

Heartbeat

Add journey duration to synthetics browser events. 40230

Metricbeat

Add new metrics fot datastore and minor changes to overall vSphere metrics. 40766
Add new metricset datastorecluster for vSphere module. 40634 40694
Add AWS Cloudwatch capability to retrieve tags from AWS/ApiGateway resources. 40755
Add new metrics for the vSphere Virtualmachine metricset. 40485
Add metrics_count to Prometheus module if metrics_count: true is set. 40411

« Beats version 8.16.1 Beats version 8.15.4 »