IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Logstash 7.8.1 Release Notes Logstash 7.7.1 Release Notes »
Elastic Docs ›Logstash Reference [7.9] ›Release Notes

Logstash 7.8.0 Release Notes

edit

Logstash 7.8.0 Release Notes

edit

New features and improvements

edit

Expanded JDK ecosystem and platform support

edit

We can be more flexible and responsive in supporting new JDKs and deprecating old ones, thanks to recent improvements to our test scripts infrastructure. This work and other JDK14 fixes pave the way for Logstash to support both AdoptOpenJDK 11 and 14 in the near future.

Logstash has introduced support for running on CentOS/RHEL 8.x and Ubuntu 20.04. We’ve added new JDK support for Zulu 11, AdoptOpenJDK 11, and Oracle/OpenJDK/AdoptOpenJDK 14.

JVM version info is covered in Getting Started with Logstash. The complete list of supported operating systems and JVMs is available in the support matrix.

Elasticsearch API key support

edit

Support for API keys was added to Elasticsearch in 6.7.0. With 7.8.0 Logstash introduces support for Elasticsearch API keys in the Elasticsearch output plugin #934.

Authentication in Elasticsearch can be done in different ways, from LDAP to SAML and others. User/password authentication makes sense for discrete users accessing Elasticsearch. For machine-to-machine communication, API key access is more common. Check out Grant access using API keys for more information about using API keys with Logstash and Elasticsearch.

Support for API keys in the Elasticsearch input and filter plugins, and the monitoring and management features will be added in upcoming releases.

Proxy support for monitoring and centralized management

edit

Many of our users deploy Logstash and the Elastic Stack in segmented networks where one component may not be able to directly reach out to another or to the Internet. Logstash plugins, such as the elasticsearch, http and SNS outputs, support the configuration of proxy servers. Version 7.8.0 brings proxy support to monitoring and central management #11799.

Configure the proxy’s URL in your logstash.yml file using "xpack.monitoring.elasticsearch.proxy" (for monitoring) or "xpack.management.elasticsearch.proxy" (for central management).

Performance improvements and notable issues fixed

edit
  • Performance: Share a single secret store #10794
  • Performance: Improve event.clone memory usage #11794
  • Refactor: Avoid array in case of single event #11732
  • Debugging: Print RUBY_DESCRIPTION at startup to facilitate debugging #11852
  • Fix: Avoid gsub (frame dependent) usage from Java #11874

Announcement: Azure and Netflow module deprecation

edit

Azure and Netflow modules in Logstash have been deprecated and replaced by the Azure modules in Filebeat and Metricbeat, and the Netflow module in Filebeat. The Filebeat and Metricbeat modules are compliant with the Elastic Common Schema (ECS).

Known issue

edit

Performance regression. A potential performance regression may affect some users. This issue can cause a slowdown on pipeline compilation when multiple large pipelines are in use. We believe the issue was introduced in 7.7.0. This issue is currently being tracked and investigated in #12031

This issue seems to be affecting only big pipeline installations (that is, big pipeline definitions when multiple pipelines are defined). Symptoms include increased startup time and the appearance that Logstash is not responding to input events.

If you believe this issue is affecting you, we recommended that you downgrade to 7.6.2 while we continue to investigate and provide a resolution.

Plugins

edit

Cef Codec - 6.1.1

  • Improved encoding performance, especially when encoding many extension fields #81
  • Fixed CEF short to long name translation for ahost/agentHostName field, according to documentation #75
  • Fixed support for deep dot notation #73
  • Removed obsolete sev and deprecated_v1_fields fields
  • Fixed minor doc inconsistencies (added reverse_mapping to options table, moved it to alpha order in option descriptions, fixed typo) #60
  • Added reverse_mapping option, which can be used to make encoder compliant to spec #51
  • Fix handling of malformed inputs that have illegal unescaped-equals characters in extension field values (restores behaviour from ⇐ v5.0.3 in some edge-cases) #56
  • Fix bug in parsing headers where certain legal escape sequences could cause non-escaped pipe characters to be ignored.
  • Fix bug in parsing extension values where a legal unescaped space in a field’s value could be interpreted as a field separator #54
  • Add explicit handling for extension key names that use array-like syntax that isn’t legal with the strict-mode field-reference parser (e.g., fieldname[0] becomes [fieldname][0]).
  • Fix handling of higher-plane UTF-8 characters in message body
  • move sev and deprecated_v1_fields fields from deprecated to obsolete
  • added mapping for outcome = eventOutcome from CEF whitepaper (ref:p26/39)
  • changed rt from receiptTime to deviceReceiptTime (ref:p27/39)
  • changed tokenizer to include additional fields (ad.fieldname)
  • Add delimiter setting. This allows the decoder to be used with inputs like the TCP input where event delimiters are used.
  • Implements the dictionary translation for abbreviated CEF field names from chapter Chapter 2: ArcSight Extension Dictionary page 3 of 39 of the CEF specification.
  • add _cefparsefailure tag on failed decode
  • breaking: Updated plugin to use new Java Event APIs
  • Switch in-place sub! to sub when extracting cef_version. new Logstash Java Event does not support in-place String changes.
  • Depend on logstash-core-plugin-api instead of logstash-core, removing the need to mass update plugins on major releases of logstash
  • New dependency requirements for logstash-core for the 5.0 release
  • Implements encode with escaping according to the CEF specification
  • Config option sev is deprecated, use severity instead.
  • Plugins were updated to follow the new shutdown semantic. This allows Logstash to instruct input plugins to terminate gracefully, instead of using Thread.raise on the plugins' threads. #3895
  • Dependency on logstash-core update to 2.0

Elasticsearch Filter - 3.7.1

  • Fix: solves an issue where non-ascii unicode values in a template were not handled correctly #128

File Input - 4.1.18

  • Fix: release watched files on completion (in read-mode) #271
  • Added configuration setting check_archive_validity settings to enable gzipped files verification. Fixes: #261
  • [DOC] Added clarification for settings available with read mode #235
  • [DOC] Rearranged text and fixed formatting for mode setting #266

Syslog Input - 3.4.2

  • Remove (deprecated) dependency on thread_safe gem.
  • CI: upgrade testing #58
  • [DOC] Correct example for timezone option #53

Tcp Input - 6.0.5

  • Fix potential startup crash that could occur when multiple instances of this plugin were started simultaneously #155

Kafka Integration - 10.2.0

  • Changed: config defaults to be aligned with Kafka client defaults #30
  • updated kafka client (and its dependencies) to version 2.4.1 #16
  • added the input client_rack parameter to enable support for follower fetching
  • added the output partitioner parameter for tuning partitioning strategy
  • Refactor: normalized error logging a bit - make sure exception type is logged
  • Fix: properly handle empty ssl_endpoint_identification_algorithm #8
  • Refactor : made partition_assignment_strategy option easier to configure by accepting simple values from an enumerated set instead of requiring lengthy class paths #25

Elasticsearch Output - 10.5.1

  • [DOC] Removed outdated compatibility notices, reworked cloud notice, and fixed formatting for hosts examples #938
  • Added api_key support #934
  • [DOC] Added note about _type setting change from doc to _doc #884
  • Fixed default index value #927

File Output - 4.3.0

  • Made stale_cleanup_interval configurable #84
  • CI: upgrade testing #83
« Logstash 7.8.1 Release Notes Logstash 7.7.1 Release Notes »