Step 2: Configuring Filebeat

edit

To configure Filebeat, you edit the configuration file. For rpm and deb, you’ll find the configuration file at /etc/filebeat/filebeat.yml. For mac and win, look in the archive that you just extracted.

Here is a sample of the filebeat section of the filebeat.yml file. Filebeat uses predefined default values for most configuration options.

filebeat:
  # List of prospectors to fetch data.
  prospectors:
    # Each - is a prospector. Below are the prospector specific configurations
    -
      # Paths that should be crawled and fetched. Glob based paths.
      # For each file found under this path, a harvester is started.
      paths:
        - "/var/log/*.log"
        #- c:\programdata\elasticsearch\logs\*

      # Type of the files. Based on this the way the file is read is decided.
      # The different types cannot be mixed in one prospector
      #
      # Possible options are:
      # * log: Reads every line of the log file (default)
      # * stdin: Reads the standard in
      input_type: log

To configure Filebeat:

  1. Define the path (or paths) to your log files.

    For the most basic Filebeat configuration, you can define a single prospector with a single path. For example:

    filebeat:
      prospectors:
        -
          paths:
            - "/var/log/*.log"

    The prospector in this example harvests all files in the path /var/log/*.log, which means that Filebeat will harvest all files in the directory /var/log/ that end with .log. All patterns supported by Golang Glob are also supported here.

    To fetch all files from a predefined level of subdirectories, the following pattern can be used: /var/log/*/*.log. This fetches all .log files from the subfolders of /var/log. It does not fetch log files from the /var/log folder itself. Currently it is not possible to recursively fetch all files in all subdirectories of a directory.

  2. If you are sending output to Elasticsearch, set the IP address and port where Filebeat can find the Elasticsearch installation:

    # Configure what outputs to use when sending the data collected by the beat.
    # Multiple outputs may be used.
    output:
      ### Elasticsearch as output
      elasticsearch:
        # Array of hosts to connect to.
         hosts: ["192.168.1.42:9200"]

    If you are sending output to Logstash, see Step 3 (Optional): Configuring Filebeat to Use Logstash instead.

To test your configuration file, run Filebeat in the foreground with the following options specified: ./filebeat -configtest -e.

See Configuration Options for more details about each configuration option.