Achieving seamless API management: Introducing AWS API Gateway integration with Elastic

AWS API Gateway is a powerful service that redefines API management. It serves as a gateway for creating, deploying, and managing APIs, enabling businesses to establish seamless connections between different applications and services. With features like authentication, authorization, and traffic control, API Gateway ensures the security and reliability of API interactions.

In an era where APIs serve as the backbone of modern applications, having the means to maintain visibility and control over these vital components is absolutely essential. In this blog post, we dive deep into the comprehensive observability solution offered by Elastic^®, ensuring real-time visibility, advanced analytics, and actionable insights, empowering you to fine-tune your API Gateway for optimal performance.

For application owners and developers, this integration stands as a beacon of empowerment. Elastic's meticulous orchestration of the seamless merging of metrics, logs, and traces, built upon the robust ELK Stack foundation, equips them with potent real-time monitoring and analysis tools. These tools facilitate precise performance optimization and swift issue resolution, all within a secure and dependable environment.

With Elastic's AWS API Gateway integration, application owners and developers unlock the capability to proactively identify and resolve problems, fine-tune resource utilization, and provide extraordinary digital experiences to their users.

Architecture

Why the AWS API Gateway integration matters

API Gateway now serves as the foundation of contemporary application development, simplifying the process of creating and overseeing APIs on a large scale. Yet, monitoring and troubleshooting these API endpoints can be challenging. With the new AWS API Gateway integration introduced by Elastic, you can gain the following:

Unprecedented visibility: Monitor your API Gateway endpoints' performance, error rates, and usage metrics in real time. Get a comprehensive view of your APIs' health and performance.
Log analysis: Dive deep into API Gateway logs with ease. Our integration enables you to collect and analyze logs for HTTP, REST, and Websocket API types, helping you troubleshoot issues and gain valuable insights.
Rapid issue resolution: Identify and resolve issues in your API Gateway workflows faster than ever. Elastic Observability's powerful search and analytics tools help you pinpoint problems with ease.
Alerting and notifications: Set up custom alerts based on API Gateway metrics and logs. Receive notifications when performance thresholds are breached, ensuring that you can take action promptly.
Optimized costs: Visualize resource usage and performance metrics for your API Gateway deployments. Use these insights to optimize resource allocation and reduce operational costs.
Custom dashboards: Create customized dashboards and visualizations tailored to your API Gateway monitoring needs. Stay in control with real-time data and actionable insights.
Effortless integration: Seamlessly connect your AWS API Gateway to our observability solution. Our intuitive setup process ensures a smooth integration experience.
Scalability: Whether you have a handful of APIs or a complex API Gateway landscape, our observability solution scales to meet your needs. Grow confidently as your API infrastructure expands.

How to get started

Getting started with the AWS API Gateway integration in Elastic Observability is seamless. Here's a quick overview of the steps:

Prerequisites and configurations

If you intend to follow the steps outlined in this blog post, there are a few prerequisites and configurations that you should have in place beforehand.

You will need an account on Elastic Cloud and a deployed stack and agent. Instructions for deploying a stack on AWS can be found here. This is necessary for AWS API Gateway logging and analysis.
You will also need an AWS account with the necessary permissions to pull data from AWS. Details on the required permissions can be found in our documentation.
You can monitor API execution by using CloudWatch, which collects and processes raw data from API Gateway into readable, near-real-time metrics and logs. Details on the required steps to enable logging can be found here.

Step 1. Create an account with Elastic

Create an account on Elastic Cloud by following the steps provided.

Step 2. Add integration

Click on Add integrations. You will be navigated to a catalog of supported integrations.

Search and select AWS API Gateway.

Step 3. Configure integration

Click on the Add AWS API Gateway button and provide the required details.
If this is your first time adding an AWS integration, you’ll need to configure and enroll the Elastic Agent on an AWS instance.

Then complete the “Configure integration” form, providing all the necessary information required for agents to collect the AWS API Gateway metrics and associated CloudWatch logs. Multiple AWS credential methods are supported, including access keys, temporary security credentials, and IAM role ARN. Please see the IAM security and access documentation for more details. You can choose to collect API Gateway metrics, API Gateway logs via S3, or API Gateway logs via CloudWatch.
Click on the Save and continue button at the bottom of the page.

Step 4. Analyze and monitor

Explore the data using the out-of-the-box dashboards available for the integration. Select Discover from the Elastic Cloud top-level menu.

Or, create custom dashboards, set up alerts, and gain actionable insights into your API Gateway service performance.

Here are key monitoring metrics collected through this integration across Rest APIs, HTTP APIs, and Websocket APIs:

4XXError – The number of client-side errors captured in a given period
5XXError – The number of server-side errors captured in a given period
CacheHitCount – The number of requests served from the API cache in a given period
CacheMissCount – The number of requests served from the backend in a given period, when API caching is enabled
Count – The total number of API requests in a given period
IntegrationLatency – The time between when API Gateway relays a request to the backend and when it receives a response from the backend
Latency – The time between when API Gateway receives a request from a client and when it returns a response to the client — the latency includes the integration latency and other API Gateway overhead
DataProcessed – The amount of data processed in bytes
ConnectCount – The number of messages sent to the $connect route integration
MessageCount – The number of messages sent to the WebSocket API, either from or to the client

Conclusion

The native integration of AWS API Gateway into Elastic Observability marks a significant advancement in streamlining the monitoring and management of your APIs. With this integration, you gain access to a wealth of insights, real-time visibility, and powerful analytics tools, empowering you to optimize your API performance, enhance security, and troubleshoot with ease. Don't miss out on this opportunity to take your API management to the next level, ensuring your digital assets operate at their best, all while providing a seamless experience for your users. Embrace this integration, and stay at the forefront of API observability in the ever-evolving world of digital technology.

Visit our documentation to learn more about Elastic Observability and the AWS API Gateway integration, or contact our sales team to get started!

Start a free trial today

Start your own 7-day free trial by signing up via AWS Marketplace and quickly spin up a deployment in minutes on any of the Elastic Cloud regions on AWS around the world. Your AWS Marketplace purchase of Elastic will be included in your monthly consolidated billing statement and will draw against your committed spend with AWS.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.