- Filebeat Reference: other versions:
- Overview
- Getting Started With Filebeat
- Step 1: Install Filebeat
- Step 2: Configure Filebeat
- Step 3: Configure Filebeat to use Logstash
- Step 4: Load the index template in Elasticsearch
- Step 5: Set up the Kibana dashboards
- Step 6: Start Filebeat
- Step 7: View the sample Kibana dashboards
- Quick start: modules for common log formats
- Repositories for APT and YUM
- Setting up and running Filebeat
- Upgrading Filebeat
- How Filebeat works
- Configuring Filebeat
- Specify which modules to run
- Configure inputs
- Manage multiline messages
- Specify general settings
- Load external configuration files
- Configure the internal queue
- Configure the output
- Set up index lifecycle management
- Load balance the output hosts
- Specify SSL settings
- Filter and enhance the exported data
- Parse data by using ingest node
- Enrich events with geoIP information
- Set up project paths
- Set up the Kibana endpoint
- Load the Kibana dashboards
- Load the Elasticsearch index template
- Configure logging
- Use environment variables in the configuration
- Autodiscover
- YAML tips and gotchas
- Regular expression support
- HTTP Endpoint
- filebeat.reference.yml
- Beats central management
- Modules
- Exported fields
- Alias fields
- Apache2 fields
- Auditd fields
- Beat fields
- Cloud provider metadata fields
- Docker fields
- elasticsearch fields
- haproxy fields
- Host fields
- Icinga fields
- IIS fields
- Kafka fields
- kibana fields
- Kubernetes fields
- Log file content fields
- logstash fields
- mongodb fields
- MySQL fields
- Nginx fields
- Osquery fields
- PostgreSQL fields
- Redis fields
- System fields
- Traefik fields
- Monitoring Filebeat
- Securing Filebeat
- Troubleshooting
- Migrating from Logstash Forwarder to Filebeat
- Contributing to Beats
NOTE: You are looking at documentation for an older release. For the latest information, see the current release documentation.
DNS Reverse Lookup
editDNS Reverse Lookup
editThe DNS processor performs reverse DNS lookups of IP addresses. It caches the responses that it receives in accordance to the time-to-live (TTL) value contained in the response. It also caches failures that occur during lookups. Each instance of this processor maintains its own independent cache.
The processor uses its own DNS resolver to send requests to nameservers and does
not use the operating system’s resolver. It does not read any values contained
in /etc/hosts
.
This processor can significantly slow down your pipeline’s throughput if you have a high latency network or slow upstream nameserver. The cache will help with performance, but if the addresses being resolved have a high cardinality then the cache benefits will be diminished due to the high miss ratio.
By way of example, if each DNS lookup takes 2 milliseconds, the maximum throughput you can achieve is 500 events per second (1000 milliseconds / 2 milliseconds). If you have a high cache hit ratio then your throughput can be higher.
This is a minimal configuration example that resolves the IP addresses contained in two fields.
processors: - dns: type: reverse fields: source.ip: source.hostname destination.ip: destination.hostname
Next is a configuration example showing all options.
processors: - dns: type: reverse action: append fields: server.ip: server.hostname client.ip: client.hostname success_cache: capacity.initial: 1000 capacity.max: 10000 failure_cache: capacity.initial: 1000 capacity.max: 10000 ttl: 1m nameservers: ['192.0.2.1', '203.0.113.1'] timeout: 500ms tag_on_failure: [_dns_reverse_lookup_failed]
The dns
processor has the following configuration settings:
-
type
-
The type of DNS lookup to perform. The only supported type is
reverse
which queries for a PTR record. -
action
-
This defines the behavior of the processor when the target field
already exists in the event. The options are
append
(default) andreplace
. -
fields
- This is a mapping of source field names to target field names. The value of the source field will be used in the DNS query and result will be written to the target field.
-
success_cache.capacity.initial
-
The initial number of items that the success
cache will be allocated to hold. When initialized the processor will allocate
the memory for this number of items. Default value is
1000
. -
success_cache.capacity.max
-
The maximum number of items that the success
cache can hold. When the maximum capacity is reached a random item is evicted.
Default value is
10000
. -
failure_cache.capacity.initial
-
The initial number of items that the failure
cache will be allocated to hold. When initialized the processor will allocate
the memory for this number of items. Default value is
1000
. -
failure_cache.capacity.max
-
The maximum number of items that the failure
cache can hold. When the maximum capacity is reached a random item is evicted.
Default value is
10000
. -
failure_cache.ttl
-
The duration for which failures are cached. Valid time
units are "ns", "us" (or "µs"), "ms", "s", "m", "h". Default value is
1m
. -
nameservers
-
A list of nameservers to query. If there are multiple servers,
the resolver queries them in the order listed. If none are specified then it
will read the nameservers listed in
/etc/resolv.conf
once at initialization. On Windows you must always supply at least one nameserver. -
timeout
-
The duration after which a DNS query will timeout. This is timeout
for each DNS request so if you have 2 nameservers then the total timeout will be
2 times this value. Valid time units are "ns", "us" (or "µs"), "ms", "s", "m",
"h". Default value is
500ms
. -
tag_on_failure
- A list of tags to add to the event when any lookup fails. The tags are only added once even if multiple lookups fail. By default no tags are added upon failure.