OpenTelemetry native support

The Elastic Stack natively supports the OpenTelemetry protocol (OTLP). This means trace data and metrics collected from your applications and infrastructure can be sent directly to the Elastic Stack.

Connect your OpenTelemetry collector instances to Elastic Observability using the OTLP exporter:

receivers: 
  # ...
  otlp:

processors: 
  # ...
  memory_limiter:
    check_interval: 1s
    limit_mib: 2000
  batch:

exporters:
  debug:
    verbosity: detailed 
  otlp/elastic: 
    # Elastic APM server https endpoint without the "https://" prefix
    endpoint: "${env:ELASTIC_APM_SERVER_ENDPOINT}"  
    headers:
      # Elastic APM Server secret token
      Authorization: "Bearer ${env:ELASTIC_APM_SECRET_TOKEN}"  

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [debug, otlp/elastic]
    metrics:
      receivers: [otlp]
      exporters: [debug, otlp/elastic]
    logs: 
      receivers: [otlp]
      exporters: [debug, otlp/elastic]

You’re now ready to export traces and metrics from your services and applications.

To export traces and metrics to APM Server, instrument your services and applications with the OpenTelemetry API, SDK, or both. For example, if you are a Java developer, you need to instrument your Java app with the OpenTelemetry agent for Java. See the OpenTelemetry Instrumentation guides to download the OpenTelemetry Agent or SDK for your language.

Define environment variables to configure the OpenTelemetry agent and enable communication with Elastic APM. For example, if you are instrumenting a Java app, define the following environment variables:

export OTEL_RESOURCE_ATTRIBUTES=service.name=checkoutService,service.version=1.1,deployment.environment=production
export OTEL_EXPORTER_OTLP_ENDPOINT=https://apm_server_url:8200
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer an_apm_secret_token"
export OTEL_METRICS_EXPORTER="otlp" \
export OTEL_LOGS_EXPORTER="otlp" \ 
java -javaagent:/path/to/opentelemetry-javaagent-all.jar \
     -classpath lib/*:classes/ \
     com.mycompany.checkout.CheckoutServiceServer

You are now ready to collect traces and metrics before verifying metrics and visualizing metrics in Kibana.

APM Server supports both the OTLP/gRPC and OTLP/HTTP protocol on the same port as Elastic APM agent requests. For ease of setup, we recommend using OTLP/HTTP when proxying or load balancing requests to the APM Server.

If you use the OTLP/gRPC protocol, requests to the APM Server must use either HTTP/2 over TLS or HTTP/2 Cleartext (H2C). No matter which protocol is used, OTLP/gRPC requests will have the header: "Content-Type: application/grpc".

When using a layer 7 (L7) proxy like AWS ALB, requests must be proxied in a way that ensures requests to the APM Server follow the rules outlined above. For example, with ALB you can create rules to select an alternative backend protocol based on the headers of requests coming into ALB. In this example, you’d select the gRPC protocol when the "Content-Type: application/grpc" header exists on a request.

For more information on how to configure an AWS ALB to support gRPC, see this AWS blog post: Application Load Balancer Support for End-to-End HTTP/2 and gRPC.

For more information on how APM Server services gRPC requests, see Muxing gRPC and HTTP/1.1.