Configure Google Functions

Functionbeat runs as a Google Function on Google Cloud Platform (GCP).

Before deploying Functionbeat, you need to configure one or more functions and specify details about the services that will trigger the functions.

You configure the functions in the the functionbeat.yml configuration file. When you’re done, you can deploy the functions to your serverless environment.

The following example configures two functions: pubsub and storage. The pubsub function collects log events from Google Pub/Sub. The storage function collects log events from Google Cloud Storage. Both functions in the example forward the events to Elasticsearch.

functionbeat.provider.gcp.location_id: "europe-west2"
functionbeat.provider.gcp.project_id: "my-project-123456"
functionbeat.provider.gcp.storage_name: "functionbeat-deploy"
functionbeat.provider.gcp.functions:
  - name: pubsub
    enabled: true
    type: pubsub
    description: "Google Cloud Function for Pub/Sub"
    trigger:
      resource: "projects/_/pubsub/myPubSub"
      #service: "pubsub.googleapis.com"
  - name: storage
    enabled: true
    type: storage
    description: "Google Cloud Function for Cloud Storage"
    trigger:
      resource: "projects/my-project/buckets/my-storage"
      event_type: "google.storage.object.finalize"

cloud.id: "MyESDeployment:SomeLongString=="
cloud.auth: "elastic:mypassword"

Specify the following options to configure the functions that you want to deploy to Google Cloud Platform (GCP).

The region where your GCP project is located.

The ID of the GCP project where the function artifacts will be deployed. See the Google Cloud Function documentation to verify that Cloud Functions are supported in the region you specify.

The name of the Google Cloud storage bucket where the function artifacts will be deployed. If the bucket doesn’t exist, it will be created, if you have the correct project permissions (storage.objects.create).

A list of functions that are available for deployment.

A unique name for the Google function.

The type of GCP service to monitor. For this release, the supported types are:

A description of the function. This description is useful when you are running multiple functions and need more context about how each function is used.

The maximum amount of memory to allocate for this function. The default is 256MB.

The execution timeout in seconds. If the function does not finish in time, it is considered failed and terminated. The default is 60s. Increase this value if you see timeout messages is the Google Stackdriver logs.

The email of the service account that the function will assume as its identity. The default is {projectid}@appspot.gserviceaccount.com.email.

One or more labels to apply to the function. A label is a key-value pair that helps you organize your Google Cloud resources.

A VPC connector that the function can connect to when sending requests to resources in your VPC network.

Use the format projects/*/locations/*/connectors/* or a fully qualified URI.

The maximum instances that can be running at the same time. The default is unlimited.

The trigger that will cause the function to execute.

If true, fields with null values will be published in the output document. By default, keep_null is false.

Optional fields that you can specify to add additional information to the output. Fields can be scalar values, arrays, dictionaries, or any nested combination of these.

Define custom processors for this function. For example, you can specify a dissect processor to tokenize a string:

processors:
  - dissect:
      tokenizer: "%{key1} %{key2}"