New

The executive guide to generative AI

Read more
About usPartnersSupport|Login
IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Codec plugins Cef codec plugin »
Elastic Docs ›Logstash Reference [7.14] ›Codec plugins

Avro codec plugin

edit

Avro codec plugin

edit
  • Plugin version: v3.2.4
  • Released on: 2020-08-21
  • Changelog

For other versions, see the Versioned plugin docs.

Getting Help

edit

For questions about the plugin, open a topic in the Discuss forums. For bugs or feature requests, open an issue in Github. For the list of Elastic supported plugins, please consult the Elastic Support Matrix.

Description

edit

Read serialized Avro records as Logstash events

This plugin is used to serialize Logstash events as Avro datums, as well as deserializing Avro datums into Logstash events.

Encoding

edit

This codec is for serializing individual Logstash events as Avro datums that are Avro binary blobs. It does not encode Logstash events into an Avro file.

Decoding

edit

This codec is for deserializing individual Avro records. It is not for reading Avro files. Avro files have a unique format that must be handled upon input.

Partial deserialization

Avro format is known to support partial deserialization of arbitrary fields, providing a schema containing a subset of the schema which was used to serialize the data. This codec doesn’t support partial deserialization of arbitrary fields. Partial deserialization might work only when providing a schema which contains the first N fields of the schema used to serialize the data (and in the same order).

Usage

edit

Example usage with Kafka input.

input {
  kafka {
    codec => avro {
        schema_uri => "/tmp/schema.avsc"
    }
  }
}
filter {
  ...
}
output {
  ...
}

Avro Codec Configuration Options

edit
Setting Input type Required

schema_uri

string

Yes

tag_on_failure

boolean

No

 

schema_uri

edit
  • This is a required setting.
  • Value type is string
  • There is no default value for this setting.

schema path to fetch the schema from. This can be a http or file scheme URI example:

  • http - http://example.com/schema.avsc
  • file - /path/to/schema.avsc

tag_on_failure

edit
  • Value type is boolean
  • Default value is false

tag events with _avroparsefailure when decode fails

« Codec plugins Cef codec plugin »