avro

edit
  • Version: 3.0.0
  • Released on: 2016-09-08
  • Changelog
  • Compatible: 5.1.1.1, 5.0.0, 2.4.1, 2.4.0, 2.3.4

This plugin does not ship with Logstash by default, but it is easy to install by running bin/logstash-plugin install logstash-codec-avro.

Read serialized Avro records as Logstash events

This plugin is used to serialize Logstash events as Avro datums, as well as deserializing Avro datums into Logstash events.

Encoding

edit

This codec is for serializing individual Logstash events as Avro datums that are Avro binary blobs. It does not encode Logstash events into an Avro file.

Decoding

edit

This codec is for deserializing individual Avro records. It is not for reading Avro files. Avro files have a unique format that must be handled upon input.

Usage

edit

Example usage with Kafka input.

input {
  kafka {
    codec => avro {
        schema_uri => "/tmp/schema.avsc"
    }
  }
}
filter {
  ...
}
output {
  ...
}

 

Synopsis

edit

This plugin supports the following configuration options:

Required configuration options:

avro {
      schema_uri => ...
  }

Available configuration options:

Setting Input type Required Default value

enable_metric

boolean

No

true

id

string

No

schema_uri

string

Yes

Details

edit

 

enable_metric

edit
  • Value type is boolean
  • Default value is true

Disable or enable metric logging for this specific plugin instance by default we record all the metrics we can, but you can disable metrics collection for a specific plugin.

  • Value type is string
  • There is no default value for this setting.

Add a unique ID to the plugin instance, this ID is used for tracking information for a specific configuration of the plugin.

output {
 stdout {
   id => "ABC"
 }
}

If you don’t explicitely set this variable Logstash will generate a unique name.

schema_uri

edit
  • This is a required setting.
  • Value type is string
  • There is no default value for this setting.

schema path to fetch the schema from. This can be a http or file scheme URI example:

  • http - http://example.com/schema.avsc
  • file - /path/to/schema.avsc