Advanced configuration

The client supports many configurations options for setting up and managing connections, configuring logging, customizing the transport library, and so on.

To connect to a specific Elasticsearch host:

Elasticsearch::Client.new(host: 'search.myserver.com')

To connect to a host with specific port:

Elasticsearch::Client.new(host: 'myhost:8080')

To connect to multiple hosts:

Elasticsearch::Client.new(hosts: ['myhost1', 'myhost2'])

Instead of strings, you can pass host information as an array of Hashes:

Elasticsearch::Client.new(hosts: [ { host: 'myhost1', port: 8080 }, { host: 'myhost2', port: 8080 } ])

Common URL parts – scheme, HTTP authentication credentials, URL prefixes, and so on – are handled automatically:

Elasticsearch::Client.new(url: 'https://username:[email protected]:4430/search')

You can pass multiple URLs separated by a comma:

Elasticsearch::Client.new(urls: 'http://localhost:9200,http://localhost:9201')

Another way to configure URLs is to export the ELASTICSEARCH_URL variable.

The client is automatically going to use a round-robin algorithm across the hosts (unless you select or implement a different Connection Selector).

The default port is 9200. Specify a port for your host(s) if they differ from this default.

If you are using Elastic Cloud, the default port is port 9243. You must supply your username and password separately, and optionally a port. Refer to Elastic Cloud.

To log requests and responses to standard output with the default logger (an instance of Ruby’s ::Logger class), set the log argument to true:

Elasticsearch::Client.new(log: true)

You can also use ecs-logging which is a set of libraries that enables you to transform your application logs to structured logs that comply with the Elastic Common Schema (ECS):

logger = EcsLogging::Logger.new($stdout)
Elasticsearch::Client.new(logger: logger)

To trace requests and responses in the Curl format, set the trace argument:

Elasticsearch::Client.new(trace: true)

You can customize the default logger or tracer:

client.transport.logger.formatter = proc { |s, d, p, m| "#{s}: #{m}\n" }
client.transport.logger.level = Logger::INFO

Or, you can use a custom ::Logger instance:

Elasticsearch::Client.new(logger: Logger.new(STDERR))

You can pass the client any conforming logger implementation:

require 'logging' # https://github.com/TwP/logging/

log = Logging.logger['elasticsearch']
log.add_appenders Logging.appenders.stdout
log.level = :info

client = Elasticsearch::Client.new(logger: log)

This client integrates seamlessly with Elastic APM via the Elastic APM Agent. It automatically captures client requests if you are using the agent on your code. If you’re using elastic-apm v3.8.0 or up, you can set capture_elasticsearch_queries to true in config/elastic_apm.yml to also capture the body from requests in Elasticsearch. Refer to this example.

You can set a custom HTTP header on the client’s initializer:

client = Elasticsearch::Client.new(
  transport_options: {
    headers:
      {user_agent: "My App"}
  }
)

You can also pass in headers as a parameter to any of the API Endpoints to set custom headers for the request:

client.search(index: 'myindex', q: 'title:test', headers: {user_agent: "My App"})

The X-Opaque-Id header allows to track certain calls, or associate certain tasks with the client that started them (refer to the documentation). To use this feature, you need to set an id for opaque_id on the client on each request. Example:

client = Elasticsearch::Client.new
client.search(index: 'myindex', q: 'title:test', opaque_id: '123456')

The search request includes the following HTTP Header:

X-Opaque-Id: 123456

You can also set a prefix for X-Opaque-Id when initializing the client. This is prepended to the id you set before each request if you’re using X-Opaque-Id. Example:

client = Elasticsearch::Client.new(opaque_id_prefix: 'eu-west1_')
client.search(index: 'myindex', q: 'title:test', opaque_id: '123456')

The request includes the following HTTP Header:

X-Opaque-Id: eu-west1_123456

For many operations in Elasticsearch, the default timeouts of HTTP libraries are too low. To increase the timeout, you can use the request_timeout parameter:

Elasticsearch::Client.new(request_timeout: 5*60)

You can also use the transport_options argument documented below.

If you pass multiple hosts to the client, it rotates across them in a round-robin fashion by default. When the same client would be running in multiple processes (for example, in a Ruby web server such as Thin), it might keep connecting to the same nodes "at once". To prevent this, you can randomize the hosts collection on initialization and reloading:

Elasticsearch::Client.new(hosts: ['localhost:9200', 'localhost:9201'], randomize_hosts: true)

When the client is initialized with multiple hosts, it makes sense to retry a failed request on a different host:

Elasticsearch::Client.new(hosts: ['localhost:9200', 'localhost:9201'], retry_on_failure: true)

By default, the client retries the request 3 times. You can specify how many times to retry before it raises an exception by passing a number to retry_on_failure:

 Elasticsearch::Client.new(hosts: ['localhost:9200', 'localhost:9201'], retry_on_failure: 5)

You can also use retry_on_status to retry when specific status codes are returned:

Elasticsearch::Client.new(hosts: ['localhost:9200', 'localhost:9201'], retry_on_status: [502, 503])

These two parameters can also be used together:

Elasticsearch::Client.new(hosts: ['localhost:9200', 'localhost:9201'], retry_on_status: [502, 503], retry_on_failure: 10)

You can also set a delay_on_retry value in milliseconds. This will add a delay to wait between retries:

 Elasticsearch::Client.new(hosts: ['localhost:9200', 'localhost:9201'], retry_on_failure: 5, delay_on_retry: 1000)

Elasticsearch dynamically discovers new nodes in the cluster by default. You can leverage this in the client, and periodically check for new nodes to spread the load.

To retrieve and use the information from the Nodes Info API on every 10,000th request:

Elasticsearch::Client.new(hosts: ['localhost:9200', 'localhost:9201'], reload_connections: true)

You can pass a specific number of requests after which reloading should be performed:

Elasticsearch::Client.new(hosts: ['localhost:9200', 'localhost:9201'], reload_connections: 1_000)

To reload connections on failures, use:

Elasticsearch::Client.new(hosts: ['localhost:9200', 'localhost:9201'], reload_on_failure: true)

The reloading timeouts if not finished under 1 second by default. To change the setting:

Elasticsearch::Client.new(hosts: ['localhost:9200', 'localhost:9201'], sniffer_timeout: 3)

Elasticsearch::Client.new(
  host: 'localhost:9200',
  http: { scheme: 'https', user: 'U', password: 'P' },
  reload_connections: true,
  reload_on_failure: true
)

By default, the client rotates the connections in a round-robin fashion, using the Elasticsearch::Transport::Transport::Connections::Selector::RoundRobin strategy.

You can implement your own strategy to customize the behaviour. For example, let’s have a "rack aware" strategy, which prefers the nodes with a specific attribute. The strategy uses the other nodes, only when these are unavailable:

class RackIdSelector
  include Elasticsearch::Transport::Transport::Connections::Selector::Base

  def select(options={})
    connections.select do |c|
      # Try selecting the nodes with a `rack_id:x1` attribute first
      c.host[:attributes] && c.host[:attributes][:rack_id] == 'x1'
    end.sample || connections.to_a.sample
  end
end

Elasticsearch::Client.new hosts: ['x1.search.org', 'x2.search.org'], selector_class: RackIdSelector

By default, the MultiJSON library is used as the serializer implementation, and it picks up the "right" adapter based on gems available.

The serialization component is pluggable, though, so you can write your own by including the Elasticsearch::Transport::Transport::Serializer::Base module, implementing the required contract, and passing it to the client as the serializer_class or serializer parameter.

The library defines a number of exception classes for various client and server errors, as well as unsuccessful HTTP responses, making it possible to rescue specific exceptions with desired granularity.

The highest-level exception is Elasticsearch::Transport::Transport::Error and is raised for any generic client or server errors.

Elasticsearch::Transport::Transport::ServerError is raised for server errors only.

As an example for response-specific errors, a 404 response status raises an Elasticsearch::Transport::Transport::Errors::NotFound exception.

Finally, Elasticsearch::Transport::Transport::SnifferTimeoutError is raised when connection reloading ("sniffing") times out.