Red Hat Enterprise Linux (RHEL) and Rocky Linux

edit
Red Hat Enterprise Linux 8 (RHEL 8), 9 (RHEL 9), and Rocky Linux 8 and 9
edit

The following instructions show you how to prepare your hosts on Red Hat Enterprise Linux 8 (RHEL 8), 9 (RHEL 9), and Rocky Linux 8 and 9.

Prerequisites
edit

Create a RHEL 8 (the version must be >= 8.5), RHEL 9, Rocky Linux 8, or Rocky Linux 9 VM.

  • For RHEL 8, follow your internal guidelines to add a vanilla RHEL 8 VM to your environment. Note that the version must be >= 8.5.

Verify that required traffic is allowed. Check the Networking prerequisites and Google Cloud Platform (GCP) guidelines for a list of ports that need to be open. The technical configuration highly depends on the underlying infrastructure.

Example: For AWS, allowing traffic between hosts is implemented using security groups.

Configure the host
edit
  1. Install the OS packages lvm2, iptables, sysstat, and net-tools by executing:

    sudo dnf install lvm2 iptables sysstat net-tools 

    The ECE diagnostic script requires net-tools.

    For RHEL 9 and Rocky Linux 9, also install the containernetworking-plugins package using:

    sudo dnf -y install containernetworking-plugins
  2. Remove Docker and previously installed podman packages (if previously installed).

    sudo dnf remove docker docker-ce podman podman-remote containerd.io
  3. As a sudoers user, edit the /etc/selinux/config file:

    1. If you are not using SELinux, set it to permissive mode:

      SELINUX=permissive
    2. If you are using SELinux, set it to enforcing mode:

      Avoid customizing the host Docker path /mnt/data/docker when using SELinux. Otherwise the ECE installer script needs to be adjusted.

      SELINUX=enforcing
  4. Install podman:

    • For RHEL 8 and Rocky Linux, install version 4.*.

      sudo dnf install podman-4.* podman-remote-4.*
    • For RHEL 9, install the latest available version 4.* using dnf.

      sudo dnf install podman-4.* podman-remote-4.*
  5. [This step is for RHEL 9 and Rocky Linux 9 only] Switch the network stack from Netavark to CNI:

    1. If the /etc/containers/containers.conf file does not exist, copy the /usr/share/containers/containers.conf file to the /etc/containers/ directory (for example, using cp /usr/share/containers/containers.conf /etc/containers/).
    2. Open the /etc/containers/containers.conf file. Navigate to the network section and make sure that the network_backend setting is set to cni.
    3. Reboot the system (reboot).
    4. Check that the network stack has changed to cni:

      cat /etc/containers/containers.conf
      [...]
      [network]
      network_backend="cni"
      [...]
  6. If podman requires a proxy in your infrastructure setup, modify the /usr/share/containers/containers.conf file and add the HTTP_PROXY and HTTPS_PROXY environment variables in the [engine] section. Please note that multiple env variables in that configuration file exists — use the one in the [engine] section.

    Example:

    [engine]
    env = ["HTTP_PROXY=http://{proxy-ip}:{proxy-port}", "HTTPS_PROXY=http://{proxy-ip}:{proxy-port}"]
  7. Reload systemd configuration

    sudo systemctl daemon-reload
  8. Create OS groups, if they do not exist yet

    Reference: Users and permissions

    sudo groupadd elastic
    sudo groupadd podman
  9. Add user elastic to the podman group

    Reference: Users and permissions

    sudo useradd -g "elastic" -G "podman" elastic
  10. As a sudoers user, add the following line to /etc/sudoers.d/99-ece-users

    Reference: Users and permissions

    elastic ALL=(ALL) NOPASSWD:ALL
  11. Add the required options to the kernel boot arguments

    sudo /sbin/grubby --update-kernel=ALL --args='cgroup_enable=memory cgroup.memory=nokmem swapaccount=1'
  12. Create the directory

    sudo mkdir -p /etc/systemd/system/podman.socket.d
  13. As a sudoers user, create the file /etc/systemd/system/podman.socket.d/podman.conf with the following content. Set the correct ownership and permission.

    Both ListenStream= and ListenStream=/var/run/docker.sock parameters are required!

    File content:

    [Socket]
    ListenStream=
    ListenStream=/var/run/docker.sock
    SocketMode=770
    SocketUser=elastic
    SocketGroup=podman

    File ownership and permission:

    sudo chown root:root /etc/systemd/system/podman.socket.d/podman.conf
    sudo chmod 0644 /etc/systemd/system/podman.socket.d/podman.conf
  14. As a sudoers user, create the (text) file /usr/bin/docker with the following content. Verify that the regular double quotes in the text file are used (ASCII code Hex 22)

    #!/bin/bash
    podman-remote --url unix:///var/run/docker.sock "$@"
  15. Set the file permissions on /usr/bin/docker

    sudo chmod 0755 /usr/bin/docker
  16. As a sudoers user, add the following two lines to section [storage] in the file /etc/containers/storage.conf. Verify that those parameters are only defined once. Either remove or comment out potentially existing parameters.

    Avoid customizing the host Docker path /mnt/data/docker when using SELinux. Otherwise the ECE installer script needs to be adjusted.

    runroot = "/mnt/data/docker/runroot/"
    graphroot = "/mnt/data/docker"
  17. Enable podman so that itself and running containers start automatically after a reboot

    sudo systemctl enable podman.service
    sudo systemctl enable podman-restart.service
  18. Enable the overlay kernel module (check Use the OverlayFS storage driver) that the Podman overlay storage driver uses (check Working with the Container Storage library and tools in Red Hat Enterprise Linux).

    In the Docker world there are two overlay drivers, overlay and overlay2. Today most users use the overlay2 driver, so we just use that one, and called it overlay. Refer also to Use the OverlayFS storage driver.

    echo "overlay" | sudo tee -a /etc/modules-load.d/overlay.conf
  19. Format the additional data partition

    sudo mkfs.xfs /dev/nvme1n1
  20. Create the /mnt/data/ directory used as a mount point

    sudo install -o elastic -g elastic -d -m 700 /mnt/data
  21. As a sudoers user, modify the entry for the XFS volume in the /etc/fstab file to add pquota,prjquota. The default filesystem path used by Elastic Cloud Enterprise is /mnt/data.

    Replace /dev/nvme1n1 in the following example with the corresponding device on your host, and add this example configuration as a single line to /etc/fstab.

    /dev/nvme1n1	/mnt/data	xfs	defaults,nofail,x-systemd.automount,prjquota,pquota  0 2
  22. Restart the local-fs target

    sudo systemctl daemon-reload
    sudo systemctl restart local-fs.target
  23. Set the permissions on the newly mounted device

    ls /mnt/data
    sudo chown elastic:elastic /mnt/data
  24. Create the /mnt/data/docker directory for the Docker service storage

    Avoid customizing the host Docker path /mnt/data/docker when using SELinux. Otherwise the ECE installer script needs to be adjusted.

    sudo install -o elastic -g elastic -d -m 700 /mnt/data/docker
  25. If you want to use FirewallD, please ensure you meet the networking prerequisites. Otherwise, you can disable it with:

    sudo systemctl disable firewalld

    If FirewallD does not exist on your VM, you can skip this step.

  26. Configure kernel parameters

    cat <<EOF | sudo tee -a /etc/sysctl.conf
    # Required by Elasticsearch
    vm.max_map_count=262144
    # enable forwarding so the Docker networking works as expected
    net.ipv4.ip_forward=1
    # Decrease the maximum number of TCP retransmissions to 5 as recommended for Elasticsearch TCP retransmission timeout.
    # See https://www.elastic.co/guide/en/elasticsearch/reference/current/system-config-tcpretries.html
    net.ipv4.tcp_retries2=5
    # Make sure the host doesn't swap too early
    vm.swappiness=1
    EOF
  27. Apply the new sysctl settings

    sudo sysctl -p
    sudo systemctl restart NetworkManager
  28. As a sudoers user, adjust the system limits. Add the following configuration values to the /etc/security/limits.conf file.

    *                soft    nofile         1024000
    *                hard    nofile         1024000
    *                soft    memlock        unlimited
    *                hard    memlock        unlimited
    elastic          soft    nofile         1024000
    elastic          hard    nofile         1024000
    elastic          soft    memlock        unlimited
    elastic          hard    memlock        unlimited
    elastic          soft    nproc          unlimited
    elastic          hard    nproc          unlimited
    root             soft    nofile         1024000
    root             hard    nofile         1024000
    root             soft    memlock        unlimited
  29. NOTE: This step is optional if the Docker registry doesn’t require authentication.

    Authenticate the elastic user to pull images from the Docker registry you use, by creating the file /home/elastic/.docker/config.json. This file needs to be owned by the elastic user. If you are using a user name other than elastic, adjust the path accordingly.

    Example: In case you use docker.elastic.co, the file content looks like as follows:

    {
     "auths": {
       "docker.elastic.co": {
         "auth": "<auth-token>"
       }
     }
    }
  30. Restart the podman service by running this command:

    sudo systemctl daemon-reload
    sudo systemctl restart podman
  31. Reboot the RHEL host

    sudo reboot