Elastic connectors

edit

Elastic connectors

edit

A connector is a type of Elastic integration that syncs data from an original data source to an Elasticsearch index.

Connectors extract the original files, records, or objects; and transforms them into Elasticsearch documents.

Many connectors are available out-of-the-box on Elastic Cloud. You can also access the source code for these (and additional) connectors, and run them on your own infrastructure.

  • Native connectors are available directly within your Elastic Cloud deployment.
  • Connector clients are self-managed on your own infrastructure.

Native connectors logo cloud (managed service)

edit

Native connectors are available directly within your Elastic Cloud deployment, as a managed service. No additional infrastructure is required.

See Native connectors for details on how to configure and use native connectors.

Expand to see list of available native connectors

Connector clients (self-managed)

edit

Connector clients enable you to run connectors locally on your own infrastructure. This means you can try out new connectors before they are available natively within Elastic Cloud, and/or customize existing connectors.

See Connector clients for details on how to deploy connector clients.

Expand to see list of available connector clients

Connector framework

edit

All Elastic connectors are built using our Python connector framework. The source code is available in the elastic/connectors repository on GitHub.

The connector framework is available for developers to customize existing connector clients or build their own connectors. Refer to Elastic connector framework for details.

Connectors overview diagram

edit

The following diagram provides a high-level overview of the Elastic connectors offering and some key facts.

connectors overview

Connectors architecture for internal knowledge search

edit

The following section provides a high-level overview of common architecture approaches for the internal knowledge search use case (AKA workplace search).

Hybrid architecture

edit

Data is synced to an Elastic Cloud deployment through native connectors and/or self-managed connector clients. A self-managed search application exposes the relevant data that your end users are authorized to see in a search experience.

Summary:

  • The best combination in terms of flexibility and out-of-the box functionality
  • Integrates with Elastic Cloud hosted native connectors to bring data to Elasticsearch with minimal operational overhead
  • Connector clients allow enterprises to adhere to strict access policies when using firewalls that don’t allow incoming connections to data sources, while outgoing traffic is easier to control
  • Provides additional functionality available for connector clients such as the Extraction Service
  • Basic functionality available for Standard licenses, advanced features for Platinum licenses

The following diagram provides a high-level overview of the hybrid internal knowledge search architecture.

hybrid architecture

Self-managed architecture

edit

Data is synced to an Elastic deployment through self-managed connector clients. A self-managed search application exposes the relevant data that your end users are authorized to see in a search experience.

Summary:

  • Gives flexibility to build custom solutions tailored to specific business requirements and internal processes
  • Allows enterprises to adhere to strict access policies when using firewalls that don’t allow incoming connections to data sources, while outgoing traffic is easier to control
  • Provides additional functionality available for connector clients such as the Extraction Service
  • Feasible for air-gapped environments
  • Requires Platinum license for full spectrum of features and connector clients

The following diagram provides a high-level overview of the self-managed internal knowledge search architecture.

self managed architecture