Logstash Lines: Towards supporting NFS

By
Monica Sarbu

Welcome to Logstash Lines! With these weekly series, we're keeping you up to date with what's new in Logstash, including the latest commits and releases.

Did you know that Logstash 6.3 is already available? Try it and let us know what you think.

Progress towards supporting NFS

The team has been hard at work on the file input plugin and made improvements to file rotation handling especially when inodes are seen to "move" through files when a rename cascade file.log.N scheme is used and improvements to remote filesystem file reading; we don't read to EOF but now read up to the size as given by the remote FS client. This was verified as working by a user on NFS that previously saw ASCII NUL bytes in events.

All changes

Repository: elastic/logstash

  • fix broken classpath when whitespaces are in the path #9832
  • clean backport of #9622 that missed the 6.3 branch
Documentation

Changes in master:

  • [DOCS] Removes alternative docker pull example #9831
  • [DOCS] Clarify methods for stopping Logstash #9828

Repositories under elastic/logstash-plugins

logstash-plugins/logstash-input-file - 4.1.4

  • Fixed a regression where files discovered after first discovery were not always read from the beginning. Applies to tail mode only. #198
  • Added much better support for file rotation schemes of copy/truncate and rename cascading. Applies to tail mode only.
  • Added support for processing files over remote mounts e.g. NFS. Before, it was possible to read into memory allocated but not filled with data resulting in ASCII NUL (0) bytes in the message field. Now, files are read up to the size as given by the remote filesystem client. Applies to tail and read modes.

logstash-plugins/logstash-output-graphite - 3.1.6

  • Fixed exception handling during socket writing to prevent logstash termination #33

logstash-plugins/logstash-input-beats - 5.1.2

  • Upgrade to Gradle 4.8.1 #334
  • Explicitly set Java compiler encoding to UTF-8
  • Fix sending of acks when sequence number of batch does not start with 1 #342

Repository: elastic/logstash-docs

Changes in versioned_plugin_docs:

  • auto generated update of versioned plugin documentation #586

Changes in master:

  • updated master docs #599
  • Add azure_event_hubs input to the plug-in docs #576