- Metricbeat Reference: other versions:
- Overview
- Getting started with Metricbeat
- Setting up and running Metricbeat
- Upgrading Metricbeat
- How Metricbeat works
- Configuring Metricbeat
- Specify which modules to run
- Specify general settings
- Load external configuration files
- Configure the internal queue
- Configure the output
- Set up index lifecycle management
- Specify SSL settings
- Filter and enhance the exported data
- Parse data by using ingest node
- Enrich events with geoIP information
- Set up project paths
- Set up the Kibana endpoint
- Load the Kibana dashboards
- Load the Elasticsearch index template
- Configure logging
- Use environment variables in the configuration
- Autodiscover
- YAML tips and gotchas
- Regular expression support
- HTTP Endpoint
- metricbeat.reference.yml
- Beats central management
- Modules
- Aerospike module
- Apache module
- Ceph module
- Couchbase module
- Docker module
- Dropwizard module
- Elasticsearch module
- Elasticsearch ccr metricset
- Elasticsearch cluster_stats metricset
- Elasticsearch index metricset
- Elasticsearch index_recovery metricset
- Elasticsearch index_summary metricset
- Elasticsearch ml_job metricset
- Elasticsearch node metricset
- Elasticsearch node_stats metricset
- Elasticsearch pending_tasks metricset
- Elasticsearch shard metricset
- envoyproxy module
- Etcd module
- Golang module
- Graphite module
- HAProxy module
- HTTP module
- Jolokia module
- Kafka module
- Kibana module
- Kubernetes module
- Kubernetes apiserver metricset
- Kubernetes container metricset
- Kubernetes event metricset
- Kubernetes node metricset
- Kubernetes pod metricset
- Kubernetes state_container metricset
- Kubernetes state_deployment metricset
- Kubernetes state_node metricset
- Kubernetes state_pod metricset
- Kubernetes state_replicaset metricset
- Kubernetes state_statefulset metricset
- Kubernetes system metricset
- Kubernetes volume metricset
- kvm module
- Logstash module
- Memcached module
- MongoDB module
- Munin module
- MySQL module
- Nginx module
- PHP_FPM module
- PostgreSQL module
- Prometheus module
- RabbitMQ module
- Redis module
- System module
- System core metricset
- System cpu metricset
- System diskio metricset
- System filesystem metricset
- System fsstat metricset
- System load metricset
- System memory metricset
- System network metricset
- System process metricset
- System process_summary metricset
- System raid metricset
- System socket metricset
- System socket_summary metricset
- System uptime metricset
- traefik module
- uwsgi module
- vSphere module
- Windows module
- ZooKeeper module
- Exported fields
- Aerospike fields
- Alias fields
- Apache fields
- Beat fields
- Ceph fields
- Cloud provider metadata fields
- Common fields
- Couchbase fields
- Docker fields
- Docker fields
- Dropwizard fields
- Elasticsearch fields
- envoyproxy fields
- Etcd fields
- Golang fields
- Graphite fields
- HAProxy fields
- Host fields
- HTTP fields
- Jolokia fields
- Kafka fields
- Kibana fields
- Kubernetes fields
- Kubernetes fields
- kvm fields
- Logstash fields
- Memcached fields
- MongoDB fields
- Munin fields
- MySQL fields
- Nginx fields
- PHP_FPM fields
- PostgreSQL fields
- Prometheus fields
- RabbitMQ fields
- Redis fields
- System fields
- traefik fields
- uwsgi fields
- vSphere fields
- Windows fields
- ZooKeeper fields
- Monitoring Metricbeat
- Securing Metricbeat
- Troubleshooting
- Contributing to Beats
Frequently asked questions
editFrequently asked questions
editThis section contains frequently asked questions about Metricbeat. Also check out the Metricbeat discussion forum.
Fix for open /compat/linux/proc: no such file or directory
error on FreeBSD?
editThe system metricsets rely a Linux comparability layer to retrieve metrics on
FreeBSD. You need to mount the Linux procfs filesystem using the following
commands. You may want to add these filesystems to your /etc/fstab
so they are
mounted automatically.
sudo mount -t procfs proc /proc sudo mkdir -p /compat/linux/proc sudo mount -t linprocfs /dev/null /compat/linux/proc
Need to limit bandwidth used by Metricbeat?
editIf you need to limit bandwidth usage, we recommend that you configure the network stack on your OS to perform bandwidth throttling.
For example, the following Linux commands cap the connection between Metricbeat and Logstash by setting a limit of 50 kbps on TCP connections over port 5044:
tc qdisc add dev $DEV root handle 1: htb tc class add dev $DEV parent 1:1 classid 1:10 htb rate 50kbps ceil 50kbps tc filter add dev $DEV parent 1:0 prio 1 protocol ip handle 10 fw flowid 1:10 iptables -A OUTPUT -t mangle -p tcp --dport 5044 -j MARK --set-mark 10
Using OS tools to perform bandwidth throttling gives you better control over policies. For example, you can use OS tools to cap bandwidth during the day, but not at night. Or you can leave the bandwidth uncapped, but assign a low priority to the traffic.
Error loading config file?
editYou may encounter errors loading the config file on POSIX operating systems if:
- an unauthorized user tries to load the config file, or
- the config file has the wrong permissions.
See Config File Ownership and Permissions for more about resolving these errors.
Found Unexpected or Unknown Characters?
editEither there is a problem with the structure of your config file, or you have used a path or expression that the YAML parser cannot resolve because the config file contains characters that aren’t properly escaped.
If the YAML file contains paths with spaces or unusual characters, wrap the paths in single quotation marks (see Wrap paths in single quotation marks).
Also see the general advice under YAML tips and gotchas.
Logstash connection doesn’t work?
editYou may have configured Logstash or Metricbeat incorrectly. To resolve the issue:
-
Make sure that Logstash is running and you can connect to it. First, try to ping the Logstash host to verify that you can reach it from the host running Metricbeat. Then use either
nc
ortelnet
to make sure that the port is available. For example:ping <hostname or IP> telnet <hostname or IP> 5044
- Verify that the config file for Metricbeat specifies the correct port where Logstash is running.
- Make sure that the Elasticsearch output is commented out in the config file and the Logstash output is uncommented.
- Confirm that the most recent Beats input plugin for Logstash is installed and configured. Note that Beats will not connect to the Lumberjack input plugin. To learn how to install and update plugins, see Working with plugins.
@metadata is missing in Logstash?
editLogstash outputs remove @metadata
fields automatically. Therefore, if Logstash instances are chained directly or via some message
queue (for example, Redis or Kafka), the @metadata
field will not be available in the final Logstash instance.
To preserve @metadata
fields, use the Logstash mutate filter with the rename setting to rename the fields to
non-internal fields.
Difference between Logstash and Beats?
editBeats are lightweight data shippers that you install as agents on your servers to send specific types of operational data to Elasticsearch. Beats have a small footprint and use fewer system resources than Logstash.
Logstash has a larger footprint, but provides a broad array of input, filter, and output plugins for collecting, enriching, and transforming data from a variety of sources.
For more information, see the Logstash Introduction and the Beats Overview.
SSL client fails to connect to Logstash?
editThe host running Logstash might be unreachable or the certificate may not be valid. To resolve your issue:
-
Make sure that Logstash is running and you can connect to it. First, try to ping the Logstash host to verify that you can reach it from the host running Metricbeat. Then use either
nc
ortelnet
to make sure that the port is available. For example:ping <hostname or IP> telnet <hostname or IP> 5044
-
Verify that the certificate is valid and that the hostname and IP match.
For testing purposes only, you can set
verification_mode: none
to disable hostname checking. - Use OpenSSL to test connectivity to the Logstash server and diagnose problems. See the OpenSSL documentation for more info.
-
Make sure that you have enabled SSL (set
ssl => true
) when configuring the Beats input plugin for Logstash.
Common SSL-Related Errors and Resolutions
editHere are some common errors and ways to fix them:
x509: cannot validate certificate for <IP address> because it doesn’t contain any IP SANs
editThis happens because your certificate is only valid for the hostname present in the Subject field.
To resolve this problem, try one of these solutions:
- Create a DNS entry for the hostname mapping it to the server’s IP.
-
Create an entry in
/etc/hosts
for the hostname. Or on Windows add an entry toC:\Windows\System32\drivers\etc\hosts
. - Re-create the server certificate and add a SubjectAltName (SAN) for the IP address of the server. This make the server’s certificate valid for both the hostname and the IP address.
getsockopt: no route to host
editThis is not a SSL problem. It’s a networking problem. Make sure the two hosts can communicate.
getsockopt: connection refused
editThis is not a SSL problem. Make sure that Logstash is running and that there is no firewall blocking the traffic.
No connection could be made because the target machine actively refused it
editA firewall is refusing the connection. Check if a firewall is blocking the traffic on the client, the network, or the destination host.
Monitoring UI shows fewer Beats than expected
editIf you are running multiple Beat instances on the same host, make sure they each have a distinct path.data
value.
On this page
- Fix for
open /compat/linux/proc: no such file or directory
error on FreeBSD? - Need to limit bandwidth used by Metricbeat?
- Error loading config file?
- Found Unexpected or Unknown Characters?
- Logstash connection doesn’t work?
- @metadata is missing in Logstash?
- Difference between Logstash and Beats?
- SSL client fails to connect to Logstash?
- Common SSL-Related Errors and Resolutions
- x509: cannot validate certificate for <IP address> because it doesn’t contain any IP SANs
- getsockopt: no route to host
- getsockopt: connection refused
- No connection could be made because the target machine actively refused it
- Monitoring UI shows fewer Beats than expected