- Elastic Cloud on Kubernetes:
- Overview
- Quickstart
- Operating ECK
- Orchestrating Elastic Stack applications
- Run Elasticsearch on ECK
- Node configuration
- Volume claim templates
- Storage recommendations
- Transport settings
- Virtual memory
- Settings managed by ECK
- Secure settings
- Custom configuration files and plugins
- Init containers for plugin downloads
- Update strategy
- Pod disruption budget
- Nodes orchestration
- Advanced Elasticsearch node scheduling
- Create automated snapshots
- Remote clusters
- Readiness probe
- Pod PreStop hook
- Elasticsearch autoscaling
- JVM heap dumps
- Security Context
- Run Kibana on ECK
- Run APM Server on ECK
- Run standalone Elastic Agent on ECK
- Run Fleet-managed Elastic Agent on ECK
- Run Elastic Maps Server on ECK
- Run Enterprise Search on ECK
- Run Beats on ECK
- Run Logstash on ECK
- Elastic Stack Helm Chart
- Recipes
- Secure the Elastic Stack
- Access Elastic Stack services
- Customize Pods
- Manage compute resources
- Autoscaling stateless applications
- Elastic Stack configuration policies
- Upgrade the Elastic Stack version
- Connect to external Elastic resources
- Run Elasticsearch on ECK
- Advanced topics
- Troubleshooting ECK
- Reference
- API Reference
- agent.k8s.elastic.co/v1alpha1
- apm.k8s.elastic.co/v1
- apm.k8s.elastic.co/v1beta1
- autoscaling.k8s.elastic.co/v1alpha1
- beat.k8s.elastic.co/v1beta1
- common.k8s.elastic.co/v1
- common.k8s.elastic.co/v1alpha1
- common.k8s.elastic.co/v1beta1
- elasticsearch.k8s.elastic.co/v1
- elasticsearch.k8s.elastic.co/v1beta1
- enterprisesearch.k8s.elastic.co/v1
- enterprisesearch.k8s.elastic.co/v1beta1
- kibana.k8s.elastic.co/v1
- kibana.k8s.elastic.co/v1beta1
- logstash.k8s.elastic.co/v1alpha1
- maps.k8s.elastic.co/v1alpha1
- stackconfigpolicy.k8s.elastic.co/v1alpha1
- Glossary
- Third-party dependencies
- API Reference
- Release highlights
- 2.16.0 release highlights
- 2.15.0 release highlights
- 2.14.0 release highlights
- 2.13.0 release highlights
- 2.12.1 release highlights
- 2.12.0 release highlights
- 2.11.1 release highlights
- 2.11.0 release highlights
- 2.10.0 release highlights
- 2.9.0 release highlights
- 2.8.0 release highlights
- 2.7.0 release highlights
- 2.6.2 release highlights
- 2.6.1 release highlights
- 2.6.0 release highlights
- 2.5.0 release highlights
- 2.4.0 release highlights
- 2.3.0 release highlights
- 2.2.0 release highlights
- 2.1.0 release highlights
- 2.0.0 release highlights
- 1.9.1 release highlights
- 1.9.0 release highlights
- 1.8.0 release highlights
- 1.7.1 release highlights
- 1.7.0 release highlights
- 1.6.0 release highlights
- 1.5.0 release highlights
- 1.4.1 release highlights
- 1.4.0 release highlights
- 1.3.2 release highlights
- 1.3.1 release highlights
- 1.3.0 release highlights
- 1.2.2 release highlights
- 1.2.1 release highlights
- 1.2.0 release highlights
- 1.1.2 release highlights
- 1.1.1 release highlights
- 1.1.0 release highlights
- 1.0.1 release highlights
- 1.0.0 release highlights
- 1.0.0-beta1 release highlights
- Release notes
- Elastic Cloud on Kubernetes version 2.16.0
- Elastic Cloud on Kubernetes version 2.15.0
- Elastic Cloud on Kubernetes version 2.14.0
- Elastic Cloud on Kubernetes version 2.13.0
- Elastic Cloud on Kubernetes version 2.12.1
- Elastic Cloud on Kubernetes version 2.12.0
- Elastic Cloud on Kubernetes version 2.11.1
- Elastic Cloud on Kubernetes version 2.11.0
- Elastic Cloud on Kubernetes version 2.10.0
- Elastic Cloud on Kubernetes version 2.9.0
- Elastic Cloud on Kubernetes version 2.8.0
- Elastic Cloud on Kubernetes version 2.7.0
- Elastic Cloud on Kubernetes version 2.6.2
- Elastic Cloud on Kubernetes version 2.6.1
- Elastic Cloud on Kubernetes version 2.6.0
- Elastic Cloud on Kubernetes version 2.5.0
- Elastic Cloud on Kubernetes version 2.4.0
- Elastic Cloud on Kubernetes version 2.3.0
- Elastic Cloud on Kubernetes version 2.2.0
- Elastic Cloud on Kubernetes version 2.1.0
- Elastic Cloud on Kubernetes version 2.0.0
- Elastic Cloud on Kubernetes version 1.9.1
- Elastic Cloud on Kubernetes version 1.9.0
- Elastic Cloud on Kubernetes version 1.8.0
- Elastic Cloud on Kubernetes version 1.7.1
- Elastic Cloud on Kubernetes version 1.7.0
- Elastic Cloud on Kubernetes version 1.6.0
- Elastic Cloud on Kubernetes version 1.5.0
- Elastic Cloud on Kubernetes version 1.4.1
- Elastic Cloud on Kubernetes version 1.4.0
- Elastic Cloud on Kubernetes version 1.3.2
- Elastic Cloud on Kubernetes version 1.3.1
- Elastic Cloud on Kubernetes version 1.3.0
- Elastic Cloud on Kubernetes version 1.2.2
- Elastic Cloud on Kubernetes version 1.2.1
- Elastic Cloud on Kubernetes version 1.2.0
- Elastic Cloud on Kubernetes version 1.1.2
- Elastic Cloud on Kubernetes version 1.1.1
- Elastic Cloud on Kubernetes version 1.1.0
- Elastic Cloud on Kubernetes version 1.0.1
- Elastic Cloud on Kubernetes version 1.0.0
- Elastic Cloud on Kubernetes version 1.0.0-beta1
Configuration examples
editConfiguration examples
editThis section contains manifests that illustrate common use cases, and can be your starting point in exploring Logstash deployed with ECK. These manifests are self-contained and work out-of-the-box on any non-secured Kubernetes cluster. They all contain a three-node Elasticsearch cluster and a single Kibana instance.
The examples in this section are for illustration purposes only. They should not be considered production-ready.
Some of these examples use the node.store.allow_mmap: false
setting on Elasticsearch which has performance implications and should be tuned for production workloads, as described in Virtual memory.
Single pipeline defined in CRD
editkubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/config/recipes/logstash/logstash-eck.yaml
Deploys Logstash with a single pipeline defined in the CRD
Single Pipeline defined in Secret
editkubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/config/recipes/logstash/logstash-pipeline-as-secret.yaml
Deploys Logstash with a single pipeline defined in a secret, referenced by a pipelineRef
Pipeline configuration in mounted volume
editkubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/config/recipes/logstash/logstash-pipeline-as-volume.yaml
Deploys Logstash with a single pipeline defined in a secret, mounted as a volume, and referenced by
path.config
Writing to a custom Elasticsearch index
editkubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/config/recipes/logstash/logstash-es-role.yaml
Deploys Logstash and Elasticsearch, and creates an updated version of the eck_logstash_user_role
to write to a user specified index.
Creating persistent volumes for PQ and DLQ
editkubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/config/recipes/logstash/logstash-volumes.yaml
Deploys Logstash, Beats and Elasticsearch. Logstash is configured with two pipelines:
- a main pipeline for reading from the Beats instance, which will send to the DLQ if it is unable to write to Elasticsearch
-
a second pipeline, that will read from the DLQ.
In addition, persistent queues are set up.
This example shows how to configure persistent volumes outside of the default
logstash-data
persistent volume.
Elasticsearch and Kibana Stack Monitoring
editkubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/config/recipes/logstash/logstash-monitored.yaml
Deploys an Elasticsearch and Kibana monitoring cluster, and a Logstash that will send its monitoring information to this cluster. You can view the stack monitoring information in the monitoring cluster’s Kibana
Multiple pipelines/multiple Elasticsearch clusters
editkubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/2.16/config/recipes/logstash/logstash-multi.yaml
Deploys Elasticsearch in prod and qa configurations, running in separate namespaces. Logstash is configured with a multiple pipeline→pipeline configuration, with a source pipeline routing to prod
and qa
pipelines.
On this page