Run Filebeat on Cloud Foundry

edit

You can use Filebeat on Cloud Foundry to retrieve and ship logs.

Cloud Foundry credentials

edit

Filebeat needs credentials created with UAA so it can connect to loggregator to receive the logs. The uaac command will create the required credentials for connecting to loggregator.

uaac client add filebeat --name filebeat --secret changeme --authorized_grant_types client_credentials,refresh_token --authorities doppler.firehose,cloud_controller.admin_read_only

Use a unique secret: The uaac command shown here is an example. Remember to replace changeme with your secret, and update the filebeat.yml file to use your chosen secret.

Cloud Foundry deploy manifests

edit

You deploy Filebeat as an application with no route.

Cloud Foundry requires that 3 files exist inside of a directory to allow Filebeat to be pushed. The commands below provide the basic steps for getting it up and running.

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.9.3-linux-x86_64.tar.gz
tar xzvf filebeat-7.9.3-linux-x86_64.tar.gz
cd filebeat-7.9.3-linux-x86_64
curl -L -O https://raw.githubusercontent.com/elastic/beats/7.9/deploy/cloudfoundry/filebeat/filebeat.yml
# Update api_address, client_id, client_secret in downloaded filebeat.
curl -L -O https://raw.githubusercontent.com/elastic/beats/7.9/deploy/cloudfoundry/filebeat/manifest.yml

Deploy

edit

To deploy Filebeat to Cloud Foundry, run:

cf push

To check the status, run:

$ cf apps

name       requested state   instances   memory   disk   urls
filebeat   started           1/1         256M     1G

Log events should start flowing to Elasticsearch. The events are annotated with metadata added by the add_cloudfoundry_metadata processor.

Set shard_id to scale: By default Filebeat will generate a random shard_id when it starts. In the case that Filebeat needs to be scaled passed 1 instance, be sure to set a static shard_id. Not setting a static shard_id will result in duplicate events being pushed to Elasticsearch.