Automatic routing

Elastic Serverless Forwarder supports automatic routing of the following logs to the corresponding default integration data stream:

AWS CloudTrail (aws.cloudtrail)
Amazon CloudWatch (aws.cloudwatch_logs)
Elastic Load Balancing (aws.elb_logs)
AWS Network Firewall (aws.firewall_logs)
Amazon VPC Flow (aws.vpcflow)
AWS Web Application Firewall (aws.waf)

For these use cases, setting the es_datastream_name field in the configuration file is optional.

For most other use cases, you will need to set the es_datastream_name field in the configuration file to route the data to a specific data stream or index. This value should be set in the following use cases:

You want to write the data to a specific index, alias, or custom data stream, and not to the default integration data stream. This can help some users to use existing Elasticsearch assets like index templates, ingest pipelines, or dashboards, that are already set up and connected to business processes.
When using Kinesis Data Stream, CloudWatch Logs subscription filter or Direct SQS message payload inputs. Only the S3 SQS Event Notifications input method supports automatic routing to default integration data streams for several AWS service logs.
When using S3 SQS Event Notifications but where the log type is something other than AWS CloudTrail (aws.cloudtrail), Amazon CloudWatch Logs (aws.cloudwatch_logs), Elastic Load Balancing (aws.elb_logs), AWS Network Firewall (aws.firewall_logs), Amazon VPC Flow (aws.vpcflow), and AWS Web Application Firewall (aws.waf).

If the es_datastream_name is not specified, and the log cannot be matched with any of the above AWS services, then the dataset will be set to generic and the namespace set to default, pointing to the data stream name logs-generic-default.

« Route AWS service logs Use tags and filters »