Beats 7.4.0 released
We’re pleased to announce the general availability of the Beats 7.4.0 release. This is the latest stable release and is now available for download! Please refer to the release notes for the complete list of bug fixes and features.
Scalable consumption from S3 and Kafka
With Filebeat 7.4.0, we’ve introduced new ways to consume data from AWS S3 and Kafka into the Elastic Stack. This ingestion can be done scalably with at-least-once delivery guarantees.
The new S3 input in Filebeat has been added as a beta feature under the Basic license, meaning it’s free to use. It ingests raw log lines from S3 buckets by leveraging SQS queues for scalable consumption. We recommend using this Filebeat input instead of the Logstash S3 input if you are looking for a horizontally scalable solution for ingesting logs from S3.
The new Kafka input has also been added in Filebeat 7.4.0 and enables data consumption from Kafka topics. Multiple Filebeats can subscribe to the same Kafka consumer group for parallel processing from topics. Additionally, the Kafka input can be used to consume data from Azure Event Hubs given the service supports Kafka interface compatibility.
Functionbeat levels up
If you’ve been following along, Functionbeat has been going through substantial enterprise hardening for the AWS integrations over the last few releases and the 7.4.0 release continues down that path. We’ve heard clear feedback from our users and have enabled Logstash as an output for data processing with Functionbeat. Additionally, support for configurable function tags has also been added which can be taken advantage of for grouping, filtering, and cost allocation with AWS Lambda.
Most notably, Functionbeat 7.4.0 enables better support for the popular AWS cloud monitoring architecture where CloudWatch Logs are centralized through Kinesis. We’ve added a new CloudWatch Logs Kinesis function type to Functionbeat that ingests CloudWatch Logs through Kinesis in a turnkey fashion, automatically decoding and decompressing the events properly prior to shipment downstream.
This new functionality also enables multi-subscription CloudWatch monitoring where multiple CloudWatch Logs subscriptions can log to a centralized Kinesis for ingestion with Functionbeat. This is a key architecture for many monitoring AWS deployments and we encourage you to try it out!
More platforms, more data
As ubiquitous data shippers, Beats often run on host servers which can encompass a myriad of different types of operating system platforms. The more platforms Beats support means more data that they can collect, spanning from logs and metrics to wire and audit data. With Beats 7.4.0, we are pleased to announce that we have added support for RHEL 8, Amazon Linux 2, Ubuntu 18.04, and Windows Server 2019 across all of the different Beats. If you’re looking to collect data from any of these operating systems, we encourage you to take Beats for a spin to further broader the aperture of observability across your deployments.
Modules galore
The Beats platform has been a foundational driver for the broad ecosystem of modules and data sources within the Elastic Stack. In the 7.4.0 release, we’re proud to announce an abundant addition of new modules across the Observability and SIEM domains.
In the Observability realm, we’ve added a bit of everything like new Filebeat modules for IBM MQ and S3 server access logs, along with a diverse assortment of new metricsets and dashboards for various different Metricbeat modules. Check out the Elastic Infrastructure and Elastic Logs 7.4.0 blog posts for additional details!
On the SIEM front, we’ve rewritten the Auditbeat Socket dataset, added dashboards to the NetFlow module, and introduced net new modules for the Common Event Format (CEF) and Cisco Firepower Threat Defense (FTD) in Filebeat. Check out the Elastic SIEM 7.4.0 blog post for more details!
Try it out yourself!
Please download Beats 7.4.0, try it out, and let us know what you think on Twitter (@elastic) or in our forums. You can report any bugs or feature requests on the Beats Github issues page.