Ingesting data from your application
editIngesting data from your application
editThe following tutorials demonstrate how you can use the Elasticsearch language clients to ingest data from an application.
- Ingest data with Node.js on Elasticsearch Service
- Get Node.js application data securely into Elasticsearch Service, where it can then be searched and modified.
- Ingest data with Python on Elasticsearch Service
- Get Python application data securely into Elasticsearch Service, where it can then be searched and modified.
- Ingest data from Beats to Elasticsearch Service with Logstash as a proxy
- Get server metrics or other types of data from Filebeat and Metricbeat into Logstash as an intermediary, and then send that data to Elasticsearch Service. Using Logstash as a proxy limits your Elastic Stack traffic through a single, external-facing firewall exception or rule.
- Ingest data from a relational database into Elasticsearch Service
- Get data from a relational database into Elasticsearch Service using the Logstash JDBC input plugin. Logstash can be used as an efficient way to copy records and to receive updates from a relational database as changes happen, and then send the new data to a deployment.
- Ingest logs from a Python application using Filebeat
- Get logs from a Python application and deliver them securely into an Elasticsearch Service deployment. You’ll set up Filebeat to monitor an ECS-formatted log file, and then view real-time visualizations of the log events in Kibana as they occur.
- Ingest logs from a Node.js web application using Filebeat
- Get HTTP request logs from a Node.js web application and deliver them securely into an Elasticsearch Service deployment. You’ll set up Filebeat to monitor an ECS-formatted log file and then view real-time visualizations of the log events as HTTP requests occur on your Node.js web server.
You can use Elasticsearch ingest pipelines to preprocess incoming data. This enables you to optimize how your data is indexed, and simplifies tasks such as extracting error codes from a log file and mapping geographic locations to IP addresses.