- Packetbeat Reference: other versions:
- Overview
- Getting started with Packetbeat
- Setting up and running Packetbeat
- Upgrading Packetbeat
- Configuring Packetbeat
- Set traffic capturing options
- Set up flows to monitor network traffic
- Specify which transaction protocols to monitor
- Specify which processes to monitor
- Specify general settings
- Configure the internal queue
- Configure the output
- Specify SSL settings
- Filter and enhance the exported data
- Parse data by using ingest node
- Export GeoIP Information
- Set up project paths
- Set up the Kibana endpoint
- Load the Kibana dashboards
- Load the Elasticsearch index template
- Configure logging
- Use environment variables in the configuration
- YAML tips and gotchas
- HTTP Endpoint
- packetbeat.reference.yml
- Exported fields
- AMQP fields
- Beat fields
- Cassandra fields
- Cloud provider metadata fields
- Common fields
- DHCPv4 fields
- DNS fields
- Docker fields
- Flow Event fields
- Host fields
- HTTP fields
- ICMP fields
- Kubernetes fields
- Memcache fields
- MongoDb fields
- MySQL fields
- NFS fields
- PostgreSQL fields
- Raw fields
- Redis fields
- Thrift-RPC fields
- TLS fields
- Transaction Event fields
- Measurements (Transactions) fields
- Monitoring Packetbeat
- Securing Packetbeat
- Visualizing Packetbeat data in Kibana
- Troubleshooting
- Contributing to Beats
Parse data by using ingest node
editParse data by using ingest node
editWhen you use Elasticsearch for output, you can configure Packetbeat to use ingest node to pre-process documents before the actual indexing takes place in Elasticsearch. Ingest node is a convenient processing option when you want to do some extra processing on your data, but you do not require the full power of Logstash. For example, you can create an ingest node pipeline in Elasticsearch that consists of one processor that removes a field in a document followed by another processor that renames a field.
After defining the pipeline in Elasticsearch, you simply configure Packetbeat
to use the pipeline. To configure Packetbeat, you specify the pipeline ID in
the parameters
option under elasticsearch
in the packetbeat.yml
file:
output.elasticsearch: hosts: ["localhost:9200"] pipeline: my_pipeline_id
For example, let’s say that you’ve defined the following pipeline in a file
named pipeline.json
:
{ "description": "Test pipeline", "processors": [ { "lowercase": { "field": "beat.name" } } ] }
To add the pipeline in Elasticsearch, you would run:
curl -H 'Content-Type: application/json' -XPUT 'http://localhost:9200/_ingest/pipeline/test-pipeline' -d@pipeline.json
Then in the packetbeat.yml
file, you would specify:
output.elasticsearch: hosts: ["localhost:9200"] pipeline: "test-pipeline"
When you run Packetbeat, the value of beat.name
is converted to lowercase before indexing.
For more information about defining a pre-processing pipeline, see the Ingest Node documentation.