IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Logstash Introduction Installing Logstash »
Elastic Docs ›Logstash Reference [7.16]

Getting Started with Logstash

edit

Getting Started with Logstash

edit

This section guides you through the process of installing Logstash and verifying that everything is running properly. After learning how to stash your first event, you go on to create a more advanced pipeline that takes Apache web logs as input, parses the logs, and writes the parsed data to an Elasticsearch cluster. Then you learn how to stitch together multiple input and output plugins to unify data from a variety of disparate sources.

This section includes the following topics:

Java (JVM) version

edit

Logstash requires one of these versions:

  • Java 8
  • Java 11
  • Java 15 (see Using JDK 15 for settings info)

Use the official Oracle distribution or an open-source distribution, such as OpenJDK. See the Elastic Support Matrix for the official word on supported versions across releases.

Bundled JDK

Logstash offers architecture-specific downloads that include AdoptOpenJDK 11, the latest long term support (LTS) release of JDK.

Use the LS_JAVA_HOME environment variable if you want to use a JDK other than the version that is bundled. If you have the LS_JAVA_HOME environment variable set to use a custom JDK, Logstash will continue to use the JDK version you have specified, even after you upgrade.

Check your Java version

edit

Run the following command:

java -version

On systems with Java installed, this command produces output similar to the following:

java version "11.0.1" 2018-10-16 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.1+13-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.1+13-LTS, mixed mode)

LS_JAVA_HOME and JAVA_HOME

edit

Logstash uses the Java version set in LS_JAVA_HOME. The LS_JAVA_HOME environment variable must be set for Logstash to operate correctly.

If Logstash doesn’t find LS_JAVA_HOME it tries to fall back to JAVA_HOME. The usage of JAVA_HOME is now considered deprecated in favor of LS_JAVA_HOME.

On some Linux systems, you may need to have the LS_JAVA_HOME environment exported before installing Logstash, particularly if you installed Java from a tarball. Logstash uses Java during installation to automatically detect your environment and install the correct startup method (SysV init scripts, Upstart, or systemd). If Logstash is unable to find the LS_JAVA_HOME environment variable during package installation, you may get an error message, and Logstash will not start properly.

Using JDK 15

edit

Logstash supports JDK 15, but you need to update settings in jvm.options and log4j2.properties if:

  • you are upgrading from Logstash 7.11.x (or earlier) to 7.12 or later, and
  • you are using JDK 15 or later.
Updates to jvm.options
edit

In the config/jvm.options file, replace all CMS related flags with:

## GC configuration
8-13:-XX:+UseConcMarkSweepGC
8-13:-XX:CMSInitiatingOccupancyFraction=75
8-13:-XX:+UseCMSInitiatingOccupancyOnly

For more information about how to use jvm.options, please refer to JVM settings.

Updates to log4j2.properties
edit

In the config/log4j2.properties:

  • Replace properties that start with appender.rolling.avoid_pipelined_filter.* with:

    appender.rolling.avoid_pipelined_filter.type = PipelineRoutingFilter

  • Replace properties that start with appender.json_rolling.avoid_pipelined_filter.* with:

    appender.json_rolling.avoid_pipelined_filter.type = PipelineRoutingFilter

  • Replace properties that start with appender.routing.* with:

    appender.routing.type = PipelineRouting
appender.routing.name = pipeline_routing_appender
appender.routing.pipeline.type = RollingFile
appender.routing.pipeline.name = appender-${ctx:pipeline.id}
appender.routing.pipeline.fileName = ${sys:ls.logs}/pipeline_${ctx:pipeline.id}.log
appender.routing.pipeline.filePattern = ${sys:ls.logs}/pipeline_${ctx:pipeline.id}.%i.log.gz
appender.routing.pipeline.layout.type = PatternLayout
appender.routing.pipeline.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n
appender.routing.pipeline.policy.type = SizeBasedTriggeringPolicy
appender.routing.pipeline.policy.size = 100MB
appender.routing.pipeline.strategy.type = DefaultRolloverStrategy
appender.routing.pipeline.strategy.max = 30
« Logstash Introduction Installing Logstash »