IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Native connectors Workplace Search connectors »
Elastic Docs › ›Elastic connectors

Connector clients and frameworks

edit

Connector clients and frameworks

edit

Connector clients are open-code connector implementations you can deploy to your own infrastructure. Connector clients allow you to customize connectors to satisfy your needs, or use new connectors before they are available natively within Elastic Cloud.

Connector clients sync data sources directly to Elasticsearch indices. Create these indices using the Build a connector workflow within Kibana. See Connector clients and frameworks.

The following connectors are available as connector clients:

Connector clients are built using connector frameworks. Use these frameworks to deploy existing connector clients, or create your own clients to sync from custom data sources.

The following connector frameworks are available:

  • Python (branch 8.6, compatible with Elastic 8.6)
  • Ruby (branch 8.6, compatible with Elastic 8.6)

Availability and prerequisites

edit

Connector clients and frameworks are compatible with Elastic versions 8.4.0 and later.

Your Elastic deployment must include the following Elastic services:

  • Elasticsearch
  • Kibana

  • Enterprise Search with at least 4GB RAM per zone

    Connectors are not available to Elastic Cloud trial deployments. For trial deployments, the Enterprise Search service is limited to 2GB RAM.

(A new Elastic Cloud deployment includes these services by default, however, you must increase the RAM for the Enterprise Search service. See Infrastructure requirements.)

You must additionally deploy and operate one or more connector services with native mode set to false. See Connector service.

Usage

edit

Use the Build a connector workflow to create an Elasticsearch index and configure a connector client to manage the index.

build a connector workflow

Create index

edit

Create a new index to be managed by the connector.

Within Kibana, visit:

Enterprise Search > Content > Elasticsearch indices

Create a new connector index:

  1. Choose Create new index.
  2. Choose Build a connector.
  3. Provide the connector name and optionally change the language analyzer to match the human language of your data source. (The name you provide is automatically prefixed with search-.)
  4. Save your changes.

The index is created and ready to configure.

This operation requires access to Kibana and the write indices privilege for the .elastic-connectors index.

Configure connector

edit

Configure the connector to manage the index’s documents.

Continue from above, or navigate to the following location within Kibana:

Enterprise Search > Content > Elasticsearch indices

Choose the index to configure, and then choose the Configuration tab.

Configure the connector:

  1. Choose Generate API key. An API key is printed to the screen.
  2. Edit the name and description for the connector. Your team can use this information to differentiate this index from other connector indices. (These fields describe the connector and are independent of the Elasticsearch index name.)
  3. Save your changes.
  4. Locate the API key and connector ID, which are both printed to the screen. Use these values to configure the connector service you are running within your own infrastructure. Refer to the documentation for your connector service.
  5. If necessary, choose Recheck now to display additional configuration fields for the connector. Edit these fields, referring to the connector’s reference documentation as needed.
  6. Save your changes.

Optionally choose Edit sync schedule to begin managing the connector.

This operation requires access to Kibana and the write indices privilege for the .elastic-connectors index.

Manage connector

edit

To manage documents, syncs, sync rules, ingest pipelines, and other connector features, see Usage.

These processes are identical for connector clients and native connectors.

Connector service

edit

To use connector clients, you must deploy and operate one or more connector services. Connector services are available through the following connector frameworks:

  • Python (branch 8.6, compatible with Elastic 8.6)
  • Ruby (branch 8.6, compatible with Elastic 8.6)

For each connector you’d like to use, review its reference documentation to determine which connector service is required.

Deploy and operate that service with native mode set to false to use it as a connector client.

See the frameworks listed above for connector service documentation.

« Native connectors Workplace Search connectors »