IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Hot Threads API Generating Plugins »
Elastic Docs ›Logstash Reference [7.8]

Working with plugins

edit

Working with plugins

edit

Logstash has a rich collection of input, filter, codec and output plugins. Plugins are available as self-contained packages called gems and hosted on RubyGems.org. The plugin manager accessed via bin/logstash-plugin script is used to manage the lifecycle of plugins in your Logstash deployment. You can install, remove and upgrade plugins using the Command Line Interface (CLI) invocations described below.

macOS Gatekeeper warnings

Apple’s rollout of stricter notarization requirements affected the notarization of the 7.8.1 Logstash artifacts. If macOS Catalina displays a dialog when you first run Logstash that interrupts it, you will need to take an action to allow it to run. To prevent Gatekeeper checks on the Logstash files, run the following command on the downloaded .tar.gz archive or the directory to which was extracted:

xattr -d -r com.apple.quarantine <archive-or-directory>

For example, if the .tar.gz file was extracted to the default logstash-7.8.1 directory, the command is:

xattr -d -r com.apple.quarantine logstash-7.8.1

Alternatively, you can add a security override if a Gatekeeper popup appears by following the instructions in the How to open an app that hasn’t been notarized or is from an unidentified developer section of Safely open apps on your Mac.

Proxy configuration

edit

The majority of the plugin manager commands require access to the internet to reach RubyGems.org. If your organization is behind a firewall you can set these environments variables to configure Logstash to use your proxy.

export http_proxy=http://localhost:3128
export https_proxy=http://localhost:3128

Listing plugins

edit

Logstash release packages bundle common plugins so you can use them out of the box. To list the plugins currently available in your deployment:

bin/logstash-plugin list 
bin/logstash-plugin list --verbose 
bin/logstash-plugin list '*namefragment*' 
bin/logstash-plugin list --group output

Will list all installed plugins

Will list installed plugins with version information

Will list all installed plugins containing a namefragment

Will list all installed plugins for a particular group (input, filter, codec, output)

Adding plugins to your deployment

edit

The most common situation when dealing with plugin installation is when you have access to internet. Using this method, you will be able to retrieve plugins hosted on the public repository (RubyGems.org) and install on top of your Logstash installation.

bin/logstash-plugin install logstash-output-kafka

Once the plugin is successfully installed, you can start using it in your configuration file.

Advanced: Adding a locally built plugin

edit

In some cases, you want to install plugins which have not yet been released and not hosted on RubyGems.org. Logstash provides you the option to install a locally built plugin which is packaged as a ruby gem. Using a file location:

bin/logstash-plugin install /path/to/logstash-output-kafka-1.0.0.gem

Advanced: Using --path.plugins

edit

Using the Logstash --path.plugins flag, you can load a plugin source code located on your file system. Typically this is used by developers who are iterating on a custom plugin and want to test it before creating a ruby gem.

The path needs to be in a specific directory hierarchy: PATH/logstash/TYPE/NAME.rb, where TYPE is inputs filters, outputs or codecs and NAME is the name of the plugin.

# supposing the code is in /opt/shared/lib/logstash/inputs/my-custom-plugin-code.rb
bin/logstash --path.plugins /opt/shared/lib

Updating plugins

edit

Plugins have their own release cycle and are often released independent of Logstash’s core release cycle. Using the update subcommand you can get the latest version of the plugin.

bin/logstash-plugin update 
bin/logstash-plugin update logstash-output-kafka

will update all installed plugins

will update only this plugin

Removing plugins

edit

If you need to remove plugins from your Logstash installation:

bin/logstash-plugin remove logstash-output-kafka

Proxy Support

edit

The previous sections relied on Logstash being able to communicate with RubyGems.org. In certain environments, Forwarding Proxy is used to handle HTTP requests. Logstash Plugins can be installed and updated through a Proxy by setting the HTTP_PROXY environment variable:

export HTTP_PROXY=http://127.0.0.1:3128

bin/logstash-plugin install logstash-output-kafka

Once set, plugin commands install, update can be used through this proxy.

« Hot Threads API Generating Plugins »