Brewing in Beats: Rename Filebeat prospectors to inputs

By
Monica Sarbu

Did you know that Beats 6.2 is already available? Try it and let us know what you think. If you are curious to see the Beats in action, we just published the Getting Started with Beats webinar.

Do you want to learn more about the autodiscovery feature, we just released in 6.2? Last week we published a small demo that shows you the power of it.

For more details, you can watch the Docker & Kubernetes Log Collection and Monitoring with Beats and Elasticsearch webinar on Thursday, February 15, 2018 at  9 a.m. PST / 12 p.m. EST / 5 p.m. GMT/ 6 p.m. CET. 

Filebeat prospectors renamed to inputs

We have started a while ago the work of renaming “prospectors” to “inputs” all over the Filebeat codebase. With the merges from last week, the default configuration files that we provide now use input, so we can consider this complete. These code changes were designed to keep backwards compatibility, which means that the filebeat.prospectors options still work, just output a deprecation warning.

This change will be released in 6.3.

Filebeat registry optimisation

We had a case where a large registry file (over 30K files) caused a significant slow down in Filebeat due to some slow paths in the registry handling. We did several optimisations which improve the registry performance by several orders of magnitude. Note, however, that this improves the CPU time that the registry is consumes, but the disk IO can still be an issue in case of large number of files. This can be mitigated by using the filebeat.registry_flush option, which makes updating the file on disk a scheduled operation (every N seconds), instead of on every batch.

The optimisations will be released in 6.3.

Community Beat for SNMP

Otilio, created by @isalgueiro, is a community Beat that queries SNMP data. It uses the gosnmp library, and it allows the user to configure the host to query, the SNMP version, SNMP community and a list of OIDs to query.

Other changes:

Repository: elastic/beats

Metricbeat

Changes in master:

  • [metricbeat] Fix errors in process summary on latest Linux kernels #6306
  • Add filtering option by exact device names in system.diskio: diskio.include_devices. #6085

Changes in 6.2:

  • Metricbeat, include the logstash port in the metricbeat configuration #6279

Changes in 6.1:

  • Metricbeat, include the logstash port in the metricbeat configuration #6279
Packetbeat

Changes in master:

  • Fix http parsing of repeated headers #6325
Filebeat

Changes in master:

  • Optimize filebeat state ID generation #6343
  • Wrong assertions and types conversion on the logstash slow plain format. #6318
  • Update filebeat python tests to use the new field input(s) instead of prospector(s) #6284
  • Migrate: Docker prospector to the the input interface #6278
  • Fields generator: support anonymous patterns in Grok #6110
  • Improve close_timeout logging #4584
Winlogbeat

Changes in master:

  • Fix event formatting with missing params #6247
Packaging

Changes in master:

  • Fix Auditbeat yml permissions #6299
Documentation

Changes in master:

  • Add minor edits to keystore docs #6322
  • fix packetbeat devices command #6307
  • [DOCS] Monitoring Beats #6235
  • Edit autodiscover docs #6204

Changes in 6.2:

  • fix packetbeat devices command #6307
  • Mark docs as released in the 6.2 branch #6277
  • Add osquery to the 6.2 changelog #6275

Changes in 6.1:

  • fix packetbeat devices command #6307