Quickstart

edit

With Elastic Cloud on Kubernetes (ECK) you can extend the basic Kubernetes orchestration capabilities to easily deploy, secure, upgrade your Elasticsearch cluster, and much more.

Eager to get started? This fast guide shows you how to:

Requirements

Make sure that you have kubectl version 1.11+ installed.

Deploy ECK in your Kubernetes cluster

edit

If you are using GKE, make sure your user has cluster-admin permissions. For more information, see Prerequisites for using Kubernetes RBAC on GKE.

If you are using Amazon EKS, make sure the Kubernetes control plane is allowed to communicate with nodes port 443. This is required for communication with the Validating Webhook. For more information, see Recommended inbound traffic.

  1. Install custom resource definitions and the operator with its RBAC rules:

    kubectl apply -f https://download.elastic.co/downloads/eck/0.9.0/all-in-one.yaml
  2. Monitor the operator logs:

    kubectl -n elastic-system logs -f statefulset.apps/elastic-operator

Deploy the Elasticsearch cluster

edit

Apply a simple Elasticsearch cluster specification, with one node:

The default resource request is 2gb memory and 100m cpu, and your pod will be Pending if your cluster does not have enough resources.

cat <<EOF | kubectl apply -f -
apiVersion: elasticsearch.k8s.elastic.co/v1alpha1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 7.2.0
  nodes:
  - nodeCount: 1
    config:
      node.master: true
      node.data: true
      node.ingest: true
EOF

The operator automatically manages Pods and resources corresponding to the desired cluster. It may take up to a few minutes until the cluster is ready.

Monitor cluster health and creation progress

edit

Get an overview of the current Elasticsearch clusters in the Kubernetes cluster, including health, version and number of nodes:

kubectl get elasticsearch
NAME          HEALTH    NODES     VERSION   PHASE         AGE
quickstart    green     1         7.2.0     Operational   1m

When you create the cluster, there is no HEALTH status and the PHASE is Pending. After a while, the PHASE turns into Operational, and HEALTH becomes green.

You can see that one Pod is in the process of being started:

kubectl get pods --selector='elasticsearch.k8s.elastic.co/cluster-name=quickstart'
NAME                       READY     STATUS    RESTARTS   AGE
quickstart-es-5zctxpn8nd   1/1       Running   0          1m

Access the logs for that Pod:

kubectl logs -f quickstart-es-5zctxpn8nd

Request Elasticsearch access

edit

A ClusterIP Service is automatically created for your cluster:

kubectl get service quickstart-es-http
NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
quickstart-es-http   ClusterIP   10.15.251.145   <none>        9200/TCP   34m
  1. Get the credentials.

    A default user named elastic is automatically created. Its password is stored as a Kubernetes secret:

    PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o=jsonpath='{.data.elastic}' | base64 --decode)
  2. Request the Elasticsearch endpoint.

    From inside the Kubernetes cluster:

    curl -u "elastic:$PASSWORD" -k "https://quickstart-es-http:9200"

    From your local workstation, use the following command in a separate terminal:

    kubectl port-forward service/quickstart-es-http 9200

    Then request localhost:

    curl -u "elastic:$PASSWORD" -k "https://localhost:9200"

For testing purposes only, you can specify the -k option to turn off certificate verification.

{
  "name" : "quickstart-es-r56c9dzzcr",
  "cluster_name" : "quickstart",
  "cluster_uuid" : "XqWg0xIiRmmEBg4NMhnYPg",
  "version" : {...},
  "tagline" : "You Know, for Search"
}

Deploy the Kibana instance

edit

To deploy your Kibana instance go through the following steps.

  1. Specify a Kibana instance and associate it with your Elasticsearch cluster:

    cat <<EOF | kubectl apply -f -
    apiVersion: kibana.k8s.elastic.co/v1alpha1
    kind: Kibana
    metadata:
      name: quickstart
    spec:
      version: 7.2.0
      nodeCount: 1
      elasticsearchRef:
        name: quickstart
    EOF
  2. Monitor Kibana health and creation progress.

    Similarly to Elasticsearch, you can retrieve details about Kibana instances:

    kubectl get kibana

    And the associated Pods:

    kubectl get pod --selector='kibana.k8s.elastic.co/name=quickstart'
  3. Access Kibana.

    A ClusterIP Service is automatically created for Kibana:

    kubectl get service quickstart-kb-http

    Use kubectl port-forward to access Kibana from your local workstation:

    kubectl port-forward service/quickstart-kb-http 5601

    Open https://localhost:5601 in your browser. Your browser will show a warning because the self-signed certificate configured by default is not verified by a third party certificate authority and not trusted by your browser. You can either configure a valid certificate or acknowledge the warning for the purposes of this quick start.

    Login with the elastic user. Retrieve its password with:

    echo $(kubectl get secret quickstart-es-elastic-user -o=jsonpath='{.data.elastic}' | base64 --decode)

Upgrade your deployment

edit

You can apply any modification to the original cluster specification. The operator makes sure that your changes are applied to the existing cluster, while avoiding downtime.

For example, you can grow the cluster to three nodes:

cat <<EOF | kubectl apply -f -
apiVersion: elasticsearch.k8s.elastic.co/v1alpha1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 7.2.0
  nodes:
  - nodeCount: 3
    config:
      node.master: true
      node.data: true
      node.ingest: true
EOF

Update persistent storage

edit

Now that you have completed the quickstart, you can try out more features like tweaking persistent storage. The cluster that you deployed in this quickstart uses a default persistent volume claim of 1GiB, without a storage class set. This means that the default storage class defined in the Kubernetes cluster is the one that will be provisioned.

You can request a PersistentVolumeClaim with a larger size in the Elasticsearch specification or target any PersistentVolume class available in your Kubernetes cluster:

cat <<EOF | kubectl apply -f -
apiVersion: elasticsearch.k8s.elastic.co/v1alpha1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 7.2.0
  nodes:
  - nodeCount: 3
    config:
      node.master: true
      node.data: true
      node.ingest: true
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data # note: elasticsearch-data must be the name of the Elasticsearch volume
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
        #storageClassName: standard # can be any available storage class
EOF

To aim for the best performance, the operator supports persistent volumes local to each node. For more details, see:

Check out the samples

edit

You can find a set of sample resources in the project repository. To customize the Elasticsearch resource, check the Elasticsearch sample.

For a full description of each CustomResourceDefinition, go to the project repository. You can also retrieve it from the cluster. For example, describe the Elasticsearch CRD specification with:

kubectl describe crd elasticsearch