Product release

Beats 7.0.0 released

By
Monica Sarbu

We are pleased to announce the Beats 7.0.0 GA release and it's now available for download. From the highlights:

Please refer to the release notes for a complete list of bug fixes and features. 

Before upgrading from Beats 6.x, review the breaking changes list and the Beats upgrade guide.  If you are planning to upgrade the entire Elastic Stack, then don't forget to read the Elastic Stack upgrade guide. 

Elastic Common Schema (ECS)

The Elastic Common Schema, or ECS, is an open source specification that defines a common set of document fields for event data ingested into Elasticsearch. ECS field definitions have been defined to be used by most event sources. The goal of ECS is to make it dramatically easier for users to correlate data across sources and develop common content such as dashboards and machine learning jobs.

Starting with version 7.0, all Beats and Beats modules generate ECS format events by default. This means adopting ECS is as easy as upgrading to Beats 7.0. All Beats module dashboards are already modified to make use of ECS.

Migrating to a common schema means that many fields have been renamed. We have developed an upgrade procedure that will smooth out this transition, using Elasticsearch field aliases. Please make sure to review the upgrading documentation when planning your migration. Once your upgrade is done, we strongly advise adjusting your custom Kibana dashboards, machine learning jobs, and other content to use the new ECS field names. Note that new Elastic Stack users automatically benefit from ECS, as it’s the default for Beats in 7.0.

Index lifecycle management (ILM)

Starting with the 6.6 release, Elasticsearch has advanced capabilities for index management. Rather than simply performing management actions on your indices on a set schedule, you can base actions on other factors such as shard size and performance requirements. You control how indices are handled as they age by attaching a lifecycle policy to the index template used to create them. You can update the policy to modify the lifecycle of both new and existing indices. This set of capabilities are grouped in the index lifecycle management (ILM) APIs.

Starting with the 7.0 release, Beats defaults to rotating indices by using ILM policies, if the Elasticsearch version to which they connect supports ILM. The default policy rotates indices when they reach 50 GB or 30 days. You can edit the ILM policy by using the Kibana management UI, or directly via the Elasticsearch API.

Monitoring the Elastic Stack with Beats

The full suite of modules to monitor your Elastic Stack are now GA. These include the Metricbeat modules for Elasticsearch, Logstash, and Kibana. Learn more about monitoring the Elastic Stack with Beats.

The march toward Metricbeat as the recommended shipper for monitoring the Elastic Stack is accelerating. Be ahead of the curve and be prepared for the future by switching to Metricbeat to send your monitoring information by reading our guide to monitoring Elasticsearch and Kibana with Metricbeat.

Logs and Infrastructure Metrics

Beats 7.0 brings with it several new modules, focusing on datastores and the cloud.

On the cloud side we are excited to announce the new AWS EC2 module, which collects and centralizes basic resource utilization metrics from all your EC2 instances, directly from Cloudwatch.  A widely used messaging platform, NATS, earns its own module for capturing stats, connections, routes, and subscriptions metrics via the new metricsets.

The datastore metrics keep coming as well, with modules for both Microsoft SQL Server and CouchDB. MSSQL metricsets capture transaction log and performance counters, while the CouchDB provides a server metricset.

Learn more and see examples of some of the dashboards in the Infrastructure-focused release blog.

Security analytics data sources

Modules that collect data for security analytics are an area where we expect a lot of functionality growth during the 7.x series. 7.0 brings in a Filebeat module that integrates with the popular open-source Zeek project, formerly known as Bro, and a Santa Filebeat module, which can be used to track process executions on macOS. These modules add to the list of data sources already supported in the 6.x series, including Suricata, IPtables, and Netflow.

In addition, the Auditbeat system module keeps improving, and the transition to ECS makes all Beats modules so much more useful for security use-cases.

Feedback

If you want to try the new features added in Beats  7.0.0, please download it, install it, and let us know what you think on Twitter (@elastic) or in our forum. Enjoy!