Step 2: Configure Journalbeat

edit

Before running Journalbeat, you can specify the location of the systemd journal files and configure how you want the files to be read. If you accept the default configuration, Journalbeat reads from the local journal.

To configure Journalbeat, you edit the configuration file. The default configuration file is called journalbeat.yml. The location of the file varies by platform. To locate the file, see Directory layout.

There’s also a full example configuration file called journalbeat.reference.yml that shows all non-deprecated options.

See the Config File Format section of the Beats Platform Reference for more about the structure of the config file.

Here is a sample of the journalbeat section of the journalbeat.yml file. Journalbeat uses predefined default values for most configuration options.

journalbeat.inputs:
- paths: ["/path/to/journal/directory"]
  seek: head

To configure Journalbeat:

  1. Specify a list of paths to your systemd journal files. Each path can be a directory path (to collect events from all journals in a directory), or a file path. For example:

    journalbeat.inputs:
    - paths:
      - "/dev/log"
      - "/var/log/messages/my-journal-file.journal"

    If no paths are specified, Journalbeat reads from the default journal.

  2. Set the seek option to control the position where Journalbeat starts reading the journal. The available options are head, tail, and cursor. The default is cursor, which means that on first read, Journalbeat starts reading at the beginning of the file, but continues reading at the last known position after a reload or restart. For more detail about the settings, see the reference docs for the seek option.
  3. (Optional) Set the include_matches option to filter entries in journald before collecting any log events. This reduces the number of events that Journalbeat needs to process. For example, to fetch only Redis events from a Docker container tagged as redis, use:

    journalbeat.inputs:
    - paths: []
      include_matches:
        - "CONTAINER_TAG=redis"
        - "_COMM=redis"
  4. Configure the output. Journalbeat supports a variety of outputs, but typically you’ll either send events directly to Elasticsearch, or to Logstash for additional processing.

    To send output directly to Elasticsearch (without using Logstash), set the location of the Elasticsearch installation:

    • If you’re running our hosted Elasticsearch Service on Elastic Cloud, specify your Cloud ID. For example:

      cloud.id: "staging:dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRjZWM2ZjI2MWE3NGJmMjRjZTMzYmI4ODExYjg0Mjk0ZiRjNmMyY2E2ZDA0MjI0OWFmMGNjN2Q3YTllOTYyNTc0Mw=="
    • If you’re running Elasticsearch on your own hardware, set the host and port where Journalbeat can find the Elasticsearch installation. For example:

      output.elasticsearch:
        hosts: ["myEShost:9200"]

      To send output to Logstash, Configure the Logstash output instead. For all other outputs, see Configure the output.

  5. If you plan to use the sample Kibana dashboards provided with Journalbeat, configure the Kibana endpoint. You can skip this step if Kibana is running on the same host as Elasticsearch.

    setup.kibana:
      host: "mykibanahost:5601" 

    The hostname and port of the machine where Kibana is running, for example, mykibanahost:5601. If you specify a path after the port number, include the scheme and port: http://mykibanahost:5601/path.

  6. If Elasticsearch and Kibana are secured, set credentials in the journalbeat.yml config file before you run the commands that set up and start Journalbeat.

    • If you’re running our hosted Elasticsearch Service on Elastic Cloud, specify your cloud auth credentials. For example:

      cloud.auth: "elastic:YOUR_PASSWORD"
    • If you’re running Elasticsearch on your own hardware, specify your Elasticsearch and Kibana credentials:

      output.elasticsearch:
        hosts: ["myEShost:9200"]
        username: "filebeat_internal"
        password: "YOUR_PASSWORD" 
      setup.kibana:
        host: "mykibanahost:5601"
        username: "my_kibana_user"  
        password: "YOUR_PASSWORD"

      This examples shows a hard-coded password, but you should store sensitive values in the secrets keystore.

      The username and password settings for Kibana are optional. If you don’t specify credentials for Kibana, Journalbeat uses the username and password specified for the Elasticsearch output.

      To use the pre-built Kibana dashboards, this user must have the kibana_user built-in role or equivalent privileges.

      For more information, see Securing Journalbeat.

To test your configuration file, change to the directory where the Journalbeat binary is installed, and run Journalbeat in the foreground with the following options specified: ./journalbeat test config -e. Make sure your config files are in the path expected by Journalbeat (see Directory layout), or use the -c flag to specify the path to the config file.

Before starting Journalbeat, you should look at the configuration options in the configuration file. For more information about these options, see Configuring Journalbeat.