IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Using connectors Troubleshooting connectors »
Elastic Docs › ›Elastic connectors

Connector known issues

edit

Connector known issues

edit

Converting native connectors to connector clients

edit

  • Due to a bug introduced in 8.10.0, converting native connectors to connector clients is broken. (Fixed in 8.10.3)

    • One manual workaround is to find the connector document in the .elastic-connectors index and set the value for is_native to false.

Native connector configuration bug

edit

  • Due to a bug introduced in 8.10.0, some native connectors are missing configuration fields. This will lead to errors like Connector for <connector_id> has missing configuration fields: use_document_level_security.

    • This issue will be fixed in 8.11.0. We recommend upgrading to 8.11.0. Otherwise, you can manually add the missing configuration fields via the Dev Tools Console.

      To workaround the issue, you’ll follow these steps:

      1. Create a new connector with identical configuration as the existing connector.
      2. Using the Kibana Dev Tools Console, execute queries to add the missing configuration fields to the new connector.
      3. Delete the existing connector.

  • Refer to the individual references of impacted connectors for detailed workarounds:

Enterprise Search service: connector clients

edit

As of 8.10.0 self-managed connector clients no longer require the Enterprise Search service to be running on your Elastic deployment. However, if you are upgrading connectors from versions earlier than 8.9, you’ll need to run Enterprise Search once to migrate your connectors to the new format.

Some points to note about this migration:

  • This involves updating system indices that store configuration and sync history for your connectors.
  • This is an in-place operation, meaning no temporary or backup indices will be created.
  • Therefore, it is important to take a snapshot of the Elasticsearch cluster before upgrading— in the unlikely event of an index migration failure.

If you have trouble with this migration, please contact support.

To run connector clients your self-deployed connector service version must match your Elasticsearch version. For example, if you’re running Elasticsearch 8.10.1, your connector service should be version 8.10.1.x. Elastic does not support deployments running mismatched versions (except during upgrades).

Connector service

edit

The connector service has the following known issues:

  • Upgrades from deployments running on versions earlier than 8.9.0 can cause sync job failures

    Due to a bug, the job_type field mapping will be missing after upgrading from deployments running on versions earlier than 8.9.0. Sync jobs won’t be displayed in the Kibana UI (job history) and the connector service won’t be able to start new sync jobs. This will only occur if you have previously scheduled sync jobs.

    To resolve this issue, you can manually add the missing field with the following command and trigger a sync job:

    PUT .elastic-connectors-sync-jobs-v1/_mapping
{
  "properties": {
    "job_type": {
      "type": "keyword"
    }
  }
}

  • The connector service will fail to sync when the connector tries to fetch more more than 2,147,483,647 (2^31-1) documents from a data source

    A workaround is to manually partition the data to be synced using multiple search indices.

  • Custom scheduling might break when upgrading from version 8.6 or earlier.

    If you encounter the error 'custom_schedule_triggered': undefined method 'each' for nil:NilClass (NoMethodError), it means the custom scheduling feature migration failed. You can use the following manual workaround:

    POST /.elastic-connectors/_update/<connector-id>
{
  "doc": {
    "custom_scheduling": {}
  }
}

    This error can appear on Connectors or Crawlers that aren’t the cause of the issue. If the error continues, try running the above command for every document in the .elastic-connectors index.

  • Connectors upgrading from 8.7 or earlier can be missing configuration fields

    A connector that was created prior to 8.8 can sometimes be missing configuration fields. This is a known issue for the MySQL connector but could also affect other connectors.

    If the connector client raises the error Connector for <connector_id> has missing configuration fields: <field_a>, <field_b>..., you can resolve the error by manually adding the missing configuration fields via the Dev Tools. Only the following two field properties are required, as the rest will be autopopulated by the connector client:

    • type: one of str, int, bool, or list

    • value: any value, as long as it is of the correct type (list type values should be saved as comma-separated strings)

      POST /.elastic-connectors/_update/<connector_id>
{
  "doc" : {
    "configuration": {
      <field_a>: {
        "type": "str",
        "value": ""
      },
      <field_b>: {
        "type": "bool",
        "value": false
      },
      <field_c>: {
        "type": "int",
        "value": 1
      },
      <field_d>: {
        "type": "list",
        "value": "a,b"
      }
    }
  }
}

  • Python connectors that upgraded from 8.7.1 will report document volumes in gigabytes (GB) instead of megabytes (MB)

    As a result, true document volume will be under-reported by a factor of 1024.

Individual connector known issues

edit

Individual connectors may have additional known issues. Refer to each connector’s reference documentation for connector-specific known issues.

Workplace Search connector known issues

edit

For Workplace Search connectors, see the Workplace Search documentation.

« Using connectors Troubleshooting connectors »