IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

« API Authentication Reference Custom sources indexing API reference »

› ›

Content sources API reference

edit

IMPORTANT: This documentation is no longer updated. Refer to Elastic's version policy and the latest documentation.

Content sources API reference

edit

This is a technical API reference that applies to all content sources. For a conceptual walkthrough of custom sources, refer to Custom API sources.

In this API reference

edit

Content source JSON representation

edit

Content sources represent the individual connections to 3rd party services, and the documents indexed from those connections. Before using the API to create, access, update, or delete content sources, it is important to be familiar with the JSON notation that will be present in many of these response bodies.

Content source fields

edit

id (required)

The string that uniquely identifies the content source. This ID is generated by Workplace Search.

service_type (required)

The string representing the content source’s service type.

created_at (required)

The string in RFC-3339 format representing the date/time at which this content source was originally created.

last_updated_at (required)

The string in RFC-3339 format representing the date/time at which this content source was last updated.

is_remote (required)

The boolean for whether or not this content source is a "remote" content source. See Standard content sources vs remote content sources.

details (required)

The list of objects that provide metadata for the content source and the account which authenticated/connected it. These details will be displayed in the source details Overview page.

While this key is required, the list may be empty.

Any objects present in this list will contain the following keys:

title (required): The string for the human readable name of this piece of metadata.
description (required): The string value to be displayed under the corresponding title.

groups (required)

The list of objects which represent Workplace Search groups that have access to this content source. Each object contains two keypairs, one for id and one for name.

name (required)

The string for the human readable display name of this content source.

schema (conditional)

The object representing the schema that each document in a configured or custom content source must adhere to. The keys of this object can be anything constructed of lower-case letters, numbers, and the underscore (\_) character. These keys define the field names that may be present on indexed documents. The values of these keypairs must be one of text, number, date, or geolocation.

display (conditional)

The object representing the display details which govern how documents from custom content sources and configured documents from configured sources are displayed in search results.

The display object contains the following keys:

title_field (required)

The string field name that will be displayed as a search result’s title.

subtitle_field (required)

The string field name that will be displayed as a search result’s subtitle. If null, the search result will not display a subtitle.

description_field (required)

The string field name that will be displayed as a search result’s long-form description or body. In the UI, this content will be truncated to fit. If null, the search result will not have a long-form description/body.

url_field (required)

The string field name that will be used to link a given search result to its source of truth.

type_field (required)

The string field name that will be displayed as a search result’s type. If null, the search result will not display a type.

media_type_field (required)

The string field name that will be displayed as a search result’s media type. If null, the search result will not display a media type.

created_by_field (required)

The string field name that will be displayed as a search result’s creator. If null, the search result will not display a creator.

updated_by_field (required)

The string field name that will be displayed as a search result’s modifier. If null, the search result will not display a modifier.

detail_fields (required)

The list of objects representing how the detailed view of a search result item is displayed, when clicked on.

The detail_fields objects each contain the following keys:

field_name (required): The string field name that this detail pertains to.
label (required): The string for the human readable display of this field.

color (optional)

A string hex-color code. The color determines what color to fill the custom source icon with in the search UI.

color is only present for custom content sources.

context (required)

The string specifying whether this content source is available to groups of users, or a single user. The value can be either "organization" or "private."

is_searchable (required)

The boolean for whether or not this content source can currently be searched over on the search page.

indexing (conditional, optional)

An object representing the indexing configuration for the content source. Indexing configurations can be used to selectively limit the indexed content. Indexing configurations work with standard sources, excluding custom content sources. For a more detailed walkthrough, see Customizing indexing.

The indexing object contains the following keys:

enabled (required)

A boolean value determining whether this content source should run sync jobs.

If during update enabled field value is changed from true to false, all jobs except the job to update permissions (if applicable) will be interrupted.
If during update enabled field value is changed from false to true, all jobs that can run for the content source will attempt to restart.

default_action (required)

A string that defines the default indexing action. One of:

include - if no rule matches the document, it will be indexed.
exclude - if no rule matches the document, it will not be indexed.

rules (required)

An array of indexing rule objects.

Each rule object contains the following keys:

filter_type (required)

This is a string defining what document field the rule filters on. There are three possible options:

object_type - useful for the content sources that have different types of objects available to index (example: Salesforce). This rule matches against a document’s type field.
path_template - useful for the content sources that organize files into a directory structure. These include Google Drive, Dropbox, and other file-oriented sources. This rule matches against a document’s path field. Using Sharepoint Online as an example, you can identify the path value from the Workplace Search Search Application by clicking on a document of interest:

This is the document’s path field, not the full URL path to the document.
file_extension - similar to path_template, this is useful for file-based content sources. This rule matches against a document’s extension field.

One of include or exclude (required)

This is a case sensitive string defining the rule’s matching expression. For example:

for object_type filter types, a source-specific document type name. For example, lead for Salesforce.
for file_extension filter types, a file extension. For example, jpg, doc, pdf, etc. This is the bare file extension without a leading period.
for path_template filter types, a file path matching expression. Generally, path template expressiions follow the glob pattern. For example:
```
/ParentFolder/NestedFolder/**/*
/ParentFolder/\**/*.css
```

fields (conditional, optional)

For now, this only works with Salesforce or Salesforce Sandbox content sources!

This applies only to include-type rules with the object_type filter type. This is a further extension of the include-type rule. It allows you to index the objects selectively. Also, for custom object types, it allows you to index only the specific fields that you need.

The fields defined in this rule-specific fields list can have corresponding definitions in the schema and display for this content source. If there’s no field mapping in schema for a newly added field, its type will be inferred from the data.

Each field specification should have the following properties:

remote (required) - source field name;

target (required) - field name in the indexed document.

For example, with the Salesforce connector you would be able to specify a rule like this to limit the indexed data to only Id and Name from your custom object type.

{
  "rules": [
    {
      "filter_type": "object_type",
      "include": "my_awesome_object",
      "fields": [
        {
          "remote": "Id",
          "target": "id"
        },
        {
          "remote": "Name",
          "target": "name"
        }
      ]
    }
  ],
  <...>
}

schedule (required)

An object representing the sync scheduling for the content source.

The schedule object contains the following keys:

full (required)

The string (ISO 8601 Duration) representing the refresh interval for the full sync job. The default interval from yml config will be returned if it’s not a platinum license or it’s not configured.

incremental (required)

The string (ISO 8601 Duration) representing the refresh interval for the incremental sync job. The default interval from yml config will be returned if it’s not a platinum license or it’s not configured.

delete (required)

The string (ISO 8601 Duration) representing the refresh interval for the delete sync job. The default interval from yml config will be returned if it’s not a platinum license or it’s not configured.

permissions (conditional)

The string (ISO 8601 Duration) representing the refresh interval for the permissions sync job. The default interval from yml config will be returned if it’s not a platinum license or it’s not configured.

This field will only be present if the content source supports document-level-permissions and it’s enabled.

blocked_windows (conditional, optional)

An array of block window objects.

blocked_windows is only available for platinum license.

Each block window object contains the following keys:

job_type (required)

A string representing the job type. One of full, incremental, delete, permissions or all.
day (required)

A string representing the weekday. One of monday, tuesday, wednesday, thursday, friday, saturday, sunday or all.
start (optional)

A string representing the start time. It blocks from the beginning of the day if it’s not specified. start must be in HH:mm:ssZ format, e.g. 06:00:00Z.
end (optional)

A string representing the end time. It blocks until the end of the day if it’s not specified. end must be in HH:mm:ssZ format, e.g. 18:00:00Z. end must not be earlier than start.

estimates (conditional)

An object representing the estimates of sync jobs for the content source.

The field will not be present if the object is empty.

The estimates object contains the following keys:

full (conditional)

An object representing the estimates of full sync job.

The field will not be present if the object is empty.

The full object contains the following keys:
- last_run (conditional)
  
  The executed time of last full sync job.
  
  This field will not be present if there’s no past full sync job.
- duration (conditional)
  
  The average execution time for the past 5 completed full sync jobs, in ISO 8601 format.
  
  This field will not be present if there’s no past full sync job.
- next_start (conditional)
  
  The estimated execution time for the next full sync job.
  
  This field will not be present if sync is disabled for this content source or blocked for every day.
incremental (conditional)

An object representing the estimates of incremental sync job.

The field will not be present if the object is empty.

The incremental object contains the following keys:
- last_run (conditional)
  
  The executed time of last incremental sync job.
  
  This field will not be present if there’s no past incremental sync job.
- duration (conditional)
  
  The average execution time for the past 5 completed incremental sync jobs, in ISO 8601 format.
  
  This field will not be present if there’s no past incremental sync job.
- next_start (conditional)
  
  The estimated execution time for the next incremental sync job.
  
  This field will not be present if sync is disabled for this content source or blocked for every day.
delete (conditional)

An object representing the estimates of delete sync job.

The field will not be present if the object is empty.

The delete object contains the following keys:
- last_run (conditional)
  
  The executed time of last delete sync job.
  
  This field will not be present if there’s no past delete sync job.
- duration (conditional)
  
  The average execution time for the past 5 completed delete sync jobs, in ISO 8601 format.
  
  This field will not be present if there’s no past delete sync job.
- next_start (conditional)
  
  The estimated execution time for the next delete sync job.
  
  This field will not be present if sync is disabled for this content source or blocked for every day.
permissions (conditional)

An object representing the estimates of permissions sync job.

This field will only be present if the content source supports document-level-permissions and it’s enabled.

The field will not be present if the object is empty.

The permissions object contains the following keys:
- last_run (conditional)
  
  The executed time of last permissions sync job.
  
  This field will not be present if there’s no past permissions sync job.
- duration (conditional)
  
  The average execution time for the past 5 completed permissions sync jobs, in ISO 8601 format.
  
  This field will not be present if there’s no past permissions sync job.
- next_start (conditional)
  
  The estimated execution time for the next permissions sync job.
  
  This field will not be present if sync is disabled for this content source or blocked for every day.

features (optional)

An object representing features that will affect how the sync jobs will extract the content from the sources. These feature toggles can be used to extract less content, allowing to speed up some of the sync jobs and reduce the storage size required to store the data at the expense of pulling less data.

The features object contains the following keys:

content_extraction (required)

An object representing settings of content extraction feature. Full text content extraction is performed on many types of documents, including PDFs and most Office365 and GSuite formats.

The content_extraction object contains the following keys:

enabled (required):: The boolean for whether content of the document should be extracted during synchronization. Disabling this option will speed up synchronization and reduce the storage size at the expense of being unable to run full text search over the content of documents.

thumbnails (required)

An object representing settings of thumbnail creation feature. Thumbnail generation is performed on many types of documents, including PDFs, most image formats, and many Office365 and GSuite formats.

The thumbnails object contains the following keys:

enabled (required):: The boolean for whether thumbnails for the documents should be generated during synchronization. Disabling this option will speed up synchronization and reduce the storage size at the expense of not being able to see thumbnail of the documents during the search.

facets (conditional)

An object representing the facet configuration. Content source facet configuration overrides represent custom facet rules to apply when using the native search experience provided with Workplace Search. Facet configurations work with any standard content source including Custom API sources. Facet configurations will not affect the Search API.

The facets object contains the following keys:

overrides (required)

An array of facet override objects.

Each overrides object contains the following keys:

field (required)

The string that identifies the field name this facet will apply to.

enabled (required)

The boolean for whether the facet should be displayed or not. This can be used to hide facets displayed by default.

display_name (conditional)

The string to be used as a facet label in the search experience. This field is required when enabled is true.

transform (optional)

The string to describe the text transformation to be used on facet values. Valid options are:

capitalize - Ensure the first word of a facet option is capitalized. Example: white tiger becomes White tiger.
titleize - Ensure every word of a facet option is capitalized. Example: new york becomes New York.
mime_type - Parse the facet option as a raw mime type and display a more user friendly value. Example: application/pdf becomes PDF.

automatic_query_refinement (conditional)

An object representing configuration for automatic query refinement.

Determining which values exist for a given field is powered by the facet results of that field. Therefore, it is only possible to enable automatic query refinements for a field that is also enabled as a facet.

The automatic_query_refinement object contains the following keys:

overrides (required)

An array of automatic query refinement override objects.

Each overrides object contains the following keys:

field (required): The string that identifies the field name this automatic query refinement will apply to.
enabled (required): The boolean for whether the automatic query refinement for this field should be enabled or not. This can be used to disabled default automatic query refinements.
query_expansion_phrases (required): The string array that defines words/phrases that should precede a field value in order to trigger the query refinement. The empty string ("") is used to mean the value must be at the very start of the query. The empty list ([]) means that the value requires no prefix, and can be found anywhere inside the query.
is_person (required): The boolean for whether or not this field is used to uniquely identify people. This is most useful for "name" and "email" fields, and can help to resolve queries where one user alias is queried for, but another is present in a document. This helps to power queries like updated by me.

See the Automatic query refinement settings to see or change organization-wide settings for automatic query refinement.

document_count (conditional): The integer of how many documents are currently indexed in this content source. This field is not applicable to remote content sources.
last_indexed_at (conditional): The string in RFC-3339 format representing the date/time when documents were last indexed into this content source. This may be null if documents have not yet been indexed. This field is not applicable to remote content sources.

Example

edit

Below is an example content source’s JSON representation:

{
  "id": "603ebf0440b06a31115f4bad",
  "service_type": "custom",
  "created_at": "2021-03-02T22:41:08+00:00",
  "last_updated_at": "2021-03-02T22:41:16+00:00",
  "is_remote": false,
  "details": [
    {
      "title": "Cited Source",
      "description": "http://nps.gov"
    },
    {
      "title": "Usage",
      "description": "For Enterprise Search Examples"
    }
  ],
  "groups": [
    {
      "id": "603ea1a440b06a9c11fd832d",
      "name": "Default"
    }
  ],
  "name": "National Parks",
  "schema": {
    "description": "text",
    "title": "text",
    "nps_link": "text",
    "states": "text",
    "visitors": "number",
    "square_km": "number",
    "world_heritage_site": "text",
    "date_established": "date",
    "location": "geolocation",
    "acres": "number"
  },
  "display": {
    "title_field": "title",
    "subtitle_field": "states",
    "description_field": "description",
    "url_field": "nps_link",
    "type_field": null,
    "media_type_field": null,
    "created_by_field": null,
    "updated_by_field": null,
    "detail_fields": [
      {
        "field_name": "states",
        "label": "States"
      },
      {
        "field_name": "description",
        "label": "Description"
      },
      {
        "field_name": "date_established",
        "label": "Established"
      },
      {
        "field_name": "visitors",
        "label": "Number of Visitors"
      },
      {
        "field_name": "location",
        "label": "Location"
      },
      {
        "field_name": "world_heritage_site",
        "label": "World Heritage Site?"
      },
      {
        "field_name": "square_km",
        "label": "Square Kilometers"
      },
      {
        "field_name": "acres",
        "label": "Acres"
      }
    ],
    "color": "#000000"
  },
  "context": "organization",
  "is_searchable": true,
  "indexing": {
    "enabled": true,
    "default_action": "include",
    "rules": [
      {
        "filter_type": "extension",
        "exclude": "exe"
      }
    ],
    "schedule": {
      "full": "P3D",
      "incremental": "PT2H",
      "delete": "PT6H",
      "permissions": "PT5M",
      "blocked_windows": [
        {
          "job_type": "all",
          "day": "monday",
          "start": "09:00:00Z",
          "end": "17:00:00z"
       }
      ],
      "estimates": {
        "full" : {
          "last_run": "2021-06-01T23:00:00Z",
          "duration": "PT1H",
          "next_start": "2021-06-02T00:00:00Z"
        },
        "incremental" : {
          "last_run": "2021-06-01T23:55:00Z",
          "duration": "PT5M",
          "next_start": "2021-06-02T00:00:00Z"
        },
        "delete" : {
          "last_run": "2021-06-01T23:52:00Z",
          "duration": "PT8M",
          "next_start": "2021-06-02T00:00:00Z"
        },
        "permissions" : {
          "last_run": "2021-06-01T23:59:50Z",
          "duration": "PT10S",
          "next_start": "2021-06-02T00:00:00Z"
        }
      }
    },
    "features": {
      "content_extraction": {
        "enabled": true
      },
      "thumbnails": {
        "enabled": true
      }
    }
  },
  "facets": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "display_name": "State",
        "transform": "titleize"
      },
      {
        "field": "type",
        "enabled": false
      }
    ]
  },
  "automatic_query_refinement": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "query_expansion_phrases": [
          "where state is",
          "located in"
        ],
        "is_person": false
      },
      {
        "field": "type",
        "enabled": false,
        "query_expansion_phrases": [],
        "is_person": false
      }
    ]
  },
  "document_count": 59,
  "last_indexed_at": "2021-03-02T22:41:15+00:00"
}

List All content sources

edit

Retrieve a paginated list of all content sources.

GET /api/ws/v1/sources

Request headers

edit

Authorization

Bearer token permitting API access.

For more authentication methods see API Authentication Reference

Request parameters

edit

page[current] (optional): The integer page of results to retrieve. Default is 1.
page[size] (optional): The integer size of results to retrieve in a single page. Default is 25

Request examples

edit

curl \
--request 'GET' \
--url 'https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io:443/api/ws/v1/sources' \
--header "Authorization: Bearer $ACCESS_TOKEN"

curl \
--request 'GET' \
--url 'http://localhost:3002/api/ws/v1/sources' \
--header "Authorization: Bearer $ACCESS_TOKEN"

With pagination:

curl \
--request 'GET' \
--url 'http://localhost:3002/api/ws/v1/sources?page[current]=2&page[size]=1' \
--header "Authorization: Bearer $ACCESS_TOKEN"

Responses

edit

200 OK

The paginated list of content sources is returned.

{
  "meta": {
    "page": {
      "current": 1,
      "total_pages": 1,
      "total_results": 4,
      "size": 25
    }
  },
  "results": [
    { <ContentSourceOne> },
    { <ContentSourceTwo> },
    { <ContentSourceThree> },
    { <ContentSourceFour> }
  ]
}

See the above content source JSON representation.

400 Bad Request

The request body was incomplete or invalid. The response body provides additional information.

401 Unauthorized

The authorization header was missing from the request or authentication failed.

404 Not Found

The authenticated user is not an Admin user, and therefore does not have permission.

Get a content source

edit

Retrieve a single content source by its ID.

GET /api/ws/v1/sources/<content-source-id>

Request headers

edit

Authorization

Bearer token permitting API access.

For more authentication methods see API Authentication Reference

Request parameters

edit

content-source-id (required): The string ID for the content source to be retrieved.

Request examples

edit

curl \
--request 'GET' \
--url "https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io:443/api/ws/v1/sources/$CONTENT_SOURCE_ID" \
--header "Authorization: Bearer $ACCESS_TOKEN"

curl \
--request 'GET' \
--url "http://localhost:3002/api/ws/v1/sources/$CONTENT_SOURCE_ID" \
--header "Authorization: Bearer $ACCESS_TOKEN"

Responses

edit

200 OK

The single content sources is returned.

{ <ContentSource> }

See the above content source JSON representation.

401 Unauthorized

The authorization header was missing from the request or authentication failed.

404 Not Found

The authenticated user is not an Admin user, and therefore does not have permission.

The provided content_source_id does not exist.

Create a content source

edit

Create a single custom content source.

Only custom content sources can be created with this endpoint. Other service types cannot be created programmatically.

POST /api/ws/v1/sources

Request headers

edit

Authorization

Bearer token permitting API access.

For more authentication methods see API Authentication Reference

Content-Type

application/json. This endpoint accepts only JSON POST data.

Request body

edit

Content source definition (required)

The JSON object defining the content source to be created.

name (required)

The string for the human readable display name of this content source.

The name may not start or end with special characters.

The name may not include special characters other than underscore (\_).

The name may not be longer than 64 characters.

schema (optional)

The object representing the schema that each document in this content source must adhere to. The keys of this object can be anything constructed of lower-case letters, numbers, and the underscore (\_) character. These keys define the field names that may be present on indexed documents. The values of these keypairs must be one of text, number, date, or geolocation.

If omitted, you cannot specify a display at creation time.

You can update schema later by indexing documents (the schema will be inferred) or by using the update endpoint.

display (optional)

The object representing the display details which govern how documents will be displayed in search results.

The display object contains the following keys:

title_field (required)

The string field name that will be displayed as a search result’s title.

This field must exist in the schema.

url_field (required)

The string field name that will be used to link a given search result to its source of truth.

This field must exist in the schema.

subtitle_field (optional)

The string field name that will be displayed as a search result’s subtitle.

This field must exist in the schema.

If null, no field will be assigned as a subtitle, and subtitles will not be displayed for any search result in this content source.

description_field (optional)

The string field name that will be displayed as a search result’s long-form description or body. In the UI, this content will be truncated to fit.

This field must exist in the schema.

If null, no field will be assigned as a description, and descriptions will not be displayed for any search result in this content source.

type_field (optional)

The string field name that will be displayed as a search result’s type.

This field must exist in the schema.

If null, no field will be assigned as a type, and type will not be displayed for any search result in this content source.

media_type_field (optional)

The string field name that will be displayed as a search result’s media type.

This field must exist in the schema.

If null, no field will be assigned as a media type, and media type will not be displayed for any search result in this content source.

created_by_field (optional)

The string field name that will be displayed as a search result’s creator.

This field must exist in the schema.

If null, no field will be assigned as a creator, and creator will not be displayed for any search result in this content source.

updated_by_field (optional)

The string field name that will be displayed as a search result’s modifier.

This field must exist in the schema.

If null, no field will be assigned as a modifier, and modifier will not be displayed for any search result in this content source.

detail_fields (optional)

The list of objects representing how the detailed view of a search result item should be displayed when clicked on.

The detail_fields objects must each contain the following keys:

field_name (required)

The string field name that this detail pertains to.

This field must exist in the schema.

label (required)

The string for the human-readable display of this field.

color (optional)

A string hex-color code. The color determines what color to fill the custom source icon with in the search UI.

color must be a 7-character string, beginning with a hash (\#) followed by 6 hexadecimal digits (0-9, A-F). Other color strings or code formats will not be accepted.

is_searchable (optional)

The boolean for whether or not this content source should be searchable.

facets (conditional)

The facets object contains the following keys:

overrides (required)

An array of facet override objects.

Each overrides object contains the following keys:

field (required)

The string that identifies the field name this facet will apply to.

enabled (required)

The boolean for whether the facet should be displayed or not. This can be used to hide facets displayed by default.

display_name (conditional)

The string to be used as a facet label in the search experience. This field is required when enabled is true.

transform (optional)

The string to describe the text transformation to be used on facet values. Valid options are:

capitalize - Ensure the first word of a facet option is capitalized. Example: white tiger becomes White tiger.
titleize - Ensure every word of a facet option is capitalized. Example: new york becomes New York.
mime_type - Parse the facet option as a raw mime type and display a more user friendly value. Example: application/pdf becomes PDF.

automatic_query_refinement (conditional)

An object representing configuration for automatic query refinement.

The automatic_query_refinement object contains the following keys:

overrides (required)

An array of automatic query refinement override objects.

Each overrides object contains the following keys:

field (required): The string that identifies the field name this automatic query refinement will apply to.
enabled (required): The boolean for whether the automatic query refinement for this field should be enabled or not. This can be used to disabled default automatic query refinements.
query_expansion_phrases (required): The string array that defines words/phrases that should precede a field value in order to trigger the query refinement. The empty string ("") is used to mean the value must be at the very start of the query. The empty list ([]) means that the value requires no prefix, and can be found anywhere inside the query.
is_person (required): The boolean for whether or not this field is used to uniquely identify people. This is most useful for "name" and "email" fields, and can help to resolve queries where one user alias is queried for, but another is present in a document. This helps to power queries like updated by me.

See the Automatic query refinement settings to see or change organization-wide settings for automatic query refinement.

Example

edit

Below is an example content source definition’s JSON representation:

{
  "name": "National Parks",
  "schema": {
    "description": "text",
    "title": "text",
    "nps_link": "text",
    "states": "text",
    "visitors": "number",
    "square_km": "number",
    "world_heritage_site": "text",
    "date_established": "date",
    "location": "geolocation",
    "acres": "number"
  },
  "display": {
    "title_field": "title",
    "subtitle_field": "states",
    "description_field": "description",
    "url_field": "nps_link",
    "type_field": null,
    "media_type_field": null,
    "created_by_field": null,
    "updated_by_field": null,
    "detail_fields": [
      {
        "field_name": "states",
        "label": "States"
      },
      {
        "field_name": "description",
        "label": "Description"
      },
      {
        "field_name": "date_established",
        "label": "Established"
      },
      {
        "field_name": "visitors",
        "label": "Number of Visitors"
      },
      {
        "field_name": "location",
        "label": "Location"
      },
      {
        "field_name": "world_heritage_site",
        "label": "World Heritage Site?"
      },
      {
        "field_name": "square_km",
        "label": "Square Kilometers"
      },
      {
        "field_name": "acres",
        "label": "Acres"
      }
    ],
    "color": "#000000"
  },
  "is_searchable": true,
  "facets": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "display_name": "State",
        "transform": "titleize"
      },
      {
        "field": "type",
        "enabled": false
      }
    ]
  },
  "automatic_query_refinement": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "query_expansion_phrases": [
          "where state is",
          "located in"
        ],
        "is_person": false
      },
      {
        "field": "type",
        "enabled": false,
        "query_expansion_phrases": [],
        "is_person": false
      }
    ]
  }
}

Request examples

edit

curl \
--request 'POST' \
--url 'https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io:443/api/ws/v1/sources/' \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $ACCESS_TOKEN" \
--data '
{
  "name": "Test",
  "schema": {
    "title": "text",
    "url": "text",
    "my_field": "text"
  },
  "display": {
    "title_field": "title",
    "url_field": "url",
    "type_field": "my_field",
    "detail_fields": [
      {
        "field_name": "my_field",
        "label": "My Field"
      }
    ],
    "color": "#111111"
  },
  "is_searchable": true,
  "facets": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "display_name": "State",
        "transform": "titleize"
      },
      {
        "field": "type",
        "enabled": false
      }
    ]
  },
  "automatic_query_refinement": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "query_expansion_phrases": [
          "where state is",
          "located in"
        ],
        "is_person": false
      },
      {
        "field": "type",
        "enabled": false,
        "query_expansion_phrases": [],
        "is_person": false
      }
    ]
  }
}
'

curl \
--request 'POST' \
--url 'http://localhost:3002/api/ws/v1/sources/' \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $ACCESS_TOKEN" \
--data '
{
  "name": "Test",
  "schema": {
    "title": "text",
    "url": "text",
    "my_field": "text"
  },
  "display": {
    "title_field": "title",
    "url_field": "url",
    "type_field": "my_field",
    "detail_fields": [
      {
        "field_name": "my_field",
        "label": "My Field"
      }
    ],
    "color": "#111111"
  },
  "is_searchable": true,
  "facets": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "display_name": "State",
        "transform": "titleize"
      },
      {
        "field": "type",
        "enabled": false
      }
    ]
  },
  "automatic_query_refinement": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "query_expansion_phrases": [
          "where state is",
          "located in"
        ],
        "is_person": false
      },
      {
        "field": "type",
        "enabled": false,
        "query_expansion_phrases": [],
        "is_person": false
      }
    ]
  }
}
'

Responses

edit

200 OK

The single created content source is returned.

{ <ContentSource> }

See the above content source JSON representation.

400 Bad Request

The request body did not meet requirements. See the errors array to determine what may be wrong.

{
  "errors": [
    "The property '#/schema/my_field' value \"integer\" did not match one of the following values: text, geolocation, number, date",
    "The property '#/display' did not contain a required property of 'title_field'",
    "The property '#/is_searchable' did not match any of the following types: boolean",
    "The property '#/name' value \"&special_chars*\" did not match the regex '(?-mix:\\A[\\w\\s.()'\\-\\/]*\\z)'"
  ]
}

400 Invalid Display

The request body included a display configuration that, while syntactically correct, did not align with the content source’s schema. This was not detected until after the content source was created, but did prevent the display settings from being applied as expected. The response includes both the current state of the new content source, and the errors in configuring the display.

{
  "created" : { <ContentSource> },
  "errors" : [
    "Invalid display setting: title_field: \"title\" is not a field on the content source",
    "Invalid display setting: subtitle_field: \"states\" is not a field on the content source",
    "Invalid display setting: description_field: \"description\" is not a field on the content source",
    "Invalid display setting: url_field: \"nps_link\" is not a field on the content source",
    "Invalid display setting: detail_fields: \"states\" is not a field on the content source, \"description\" is not a field on the content source, \"date_established\" is not a field on the content source, \"visitors\" is not a field on the content source, \"location\" is not a field on the content source, \"world_heritage_site\" is not a field on the content source, \"square_km\" is not a field on the content source, \"acres\" is not a field on the content source"

  ]
}

See the above content source JSON representation.

401 Unauthorized

The authorization header was missing from the request or authentication failed.

404 Not Found

The authenticated user is not an Admin user, and therefore does not have permission.

Update a content source

edit

Update a single content source. Both standard and custom content sources can be updated with this endpoint.

PUT /api/ws/v1/sources/<content-source-id>

Request headers

edit

Authorization

Bearer token permitting API access.

For more authentication methods see API Authentication Reference

Content-Type

application/json. This endpoint accepts only JSON PUT data.

Request parameters

edit

content-source-id (required): The string ID for the content source to be updated.

Request body

edit

Content source definition (required)

The JSON object defining the new state of the content source.

Unlike the create request body, most parameters of this body are required. You may find it useful to first do a get request, to ensure you send the complete current values to avoid unwanted changes.

name (required)

The string for the human readable display name of this content source.

The name may not start or end with special characters.

The name may not include special characters other than underscore (\_).

The name may not be longer than 64 characters.

schema (conditional)

This field is required for custom content sources. It is ignored for standard content sources other than Salesforce.

If the content source already contains documents, this schema must not attempt to change the types of fields for document that do not have an implicit type conversion available to them. For example, a document of

{
  "field_one": 42,
  "field_two": "the answer"
}

a schema update of

{
  "field_one": "text",
  "field_two": "text"
}

is fine, because 42 (numeric) can implicitly be converted to "42" (string) without issue. However, a schema update of

{
  "field_one": "number",
  "field_two": "number"
}

will fail, because there is no implicit numeric value for "the answer".

display (conditional)

This field is required for custom content sources. It is ignored for standard content sources other than Salesforce.

The object representing the display details which govern how documents will be displayed in search results.

The display object contains the following keys:

title_field (required)

The string field name that will be displayed as a search result’s title.

This field must exist in the schema.

url_field (required)

The string field name that will be used to link a given search result to its source of truth.

This field must exist in the schema.

subtitle_field (optional)

The string field name that will be displayed as a search result’s subtitle.

This field must exist in the schema.

If null, no field will be assigned as a subtitle, and subtitles will not be displayed for any search result in this content source.

description_field (optional)

The string field name that will be displayed as a search result’s long-form description or body. In the UI, this content will be truncated to fit.

This field must exist in the schema.

If null, no field will be assigned as a description, and descriptions will not be displayed for any search result in this content source.

type_field (optional)

The string field name that will be displayed as a search result’s type.

This field must exist in the schema.

If null, no field will be assigned as a type, and type will not be displayed for any search result in this content source.

media_type_field (optional)

The string field name that will be displayed as a search result’s media type.

This field must exist in the schema.

If null, no field will be assigned as a media type, and media type will not be displayed for any search result in this content source.

created_by_field (optional)

The string field name that will be displayed as a search result’s creator.

This field must exist in the schema.

If null, no field will be assigned as a creator, and creator will not be displayed for any search result in this content source.

updated_by_field (optional)

The string field name that will be displayed as a search result’s modifier.

This field must exist in the schema.

If null, no field will be assigned as a modifier, and modifier will not be displayed for any search result in this content source.

detail_fields (optional)

The list of objects representing how the detailed view of a search result item should be displayed when clicked on.

The detail_fields objects must each contain the following keys:

field_name (required)

The string field name that this detail pertains to.

This field must exist in the schema.

label (required)

The string for the human readable display of this field.

color (required)

A string hex-color code. The color determines what color to fill the custom source icon with in the search UI.

color must be a 7-character string, beginning with a hash (\#) followed by 6 hexadecimal digits (0-9, A-F). Other color strings or code formats will not be accepted.

is_searchable (required)

The boolean for whether or not this content source should be searchable.

include::includes/indexing.asciidoc

facets (conditional)

The facets object contains the following keys:

overrides (required)

An array of facet override objects.

Each overrides object contains the following keys:

field (required)

The string that identifies the field name this facet will apply to.

enabled (required)

The boolean for whether the facet should be displayed or not. This can be used to hide facets displayed by default.

display_name (conditional)

The string to be used as a facet label in the search experience. This field is required when enabled is true.

transform (optional)

The string to describe the text transformation to be used on facet values. Valid options are:

capitalize - Ensure the first word of a facet option is capitalized. Example: white tiger becomes White tiger.
titleize - Ensure every word of a facet option is capitalized. Example: new york becomes New York.
mime_type - Parse the facet option as a raw mime type and display a more user friendly value. Example: application/pdf becomes PDF.

automatic_query_refinement (conditional)

An object representing configuration for automatic query refinement.

The automatic_query_refinement object contains the following keys:

overrides (required)

An array of automatic query refinement override objects.

Each overrides object contains the following keys:

field (required): The string that identifies the field name this automatic query refinement will apply to.
enabled (required): The boolean for whether the automatic query refinement for this field should be enabled or not. This can be used to disabled default automatic query refinements.
query_expansion_phrases (required): The string array that defines words/phrases that should precede a field value in order to trigger the query refinement. The empty string ("") is used to mean the value must be at the very start of the query. The empty list ([]) means that the value requires no prefix, and can be found anywhere inside the query.
is_person (required): The boolean for whether or not this field is used to uniquely identify people. This is most useful for "name" and "email" fields, and can help to resolve queries where one user alias is queried for, but another is present in a document. This helps to power queries like updated by me.

See the Automatic query refinement settings to see or change organization-wide settings for automatic query refinement.

Example

edit

Below is an example content source definition’s JSON representation:

{
  "name": "National Parks",
  "schema": {
    "description": "text",
    "title": "text",
    "nps_link": "text",
    "states": "text",
    "visitors": "number",
    "square_km": "number",
    "world_heritage_site": "text",
    "date_established": "date",
    "location": "geolocation",
    "acres": "number"
  },
  "display": {
    "title_field": "title",
    "subtitle_field": "states",
    "description_field": "description",
    "url_field": "nps_link",
    "type_field": null,
    "media_type_field": null,
    "created_by_field": null,
    "updated_by_field": null,
    "detail_fields": [
      {
        "field_name": "states",
        "label": "States"
      },
      {
        "field_name": "description",
        "label": "Description"
      },
      {
        "field_name": "date_established",
        "label": "Established"
      },
      {
        "field_name": "visitors",
        "label": "Number of Visitors"
      },
      {
        "field_name": "location",
        "label": "Location"
      },
      {
        "field_name": "world_heritage_site",
        "label": "World Heritage Site?"
      },
      {
        "field_name": "square_km",
        "label": "Square Kilometers"
      },
      {
        "field_name": "acres",
        "label": "Acres"
      }
    ],
    "color": "#000000"
  },
  "is_searchable": true,
  "indexing": {
    "enabled": true,
    "default_action": "include",
    "rules": [
      {
        "filter_type": "extension",
        "exclude": "exe"
      }
    ],
    "schedule": {
      "full": "P3D",
      "incremental": "PT2H",
      "delete": "PT6H",
      "permissions": "PT5M",
      "blocked_windows": [
        {
          "job_type": "all",
          "day": "monday",
          "start": "09:00:00Z",
          "end": "17:00:00z"
       }
      ]
    },
    "features": {
      "content_extraction": {
        "enabled": true
      },
      "thumbnails": {
        "enabled": true
      }
    }
  },
  "facets": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "display_name": "State",
        "transform": "titleize"
      },
      {
        "field": "type",
        "enabled": false
      }
    ]
  },
  "automatic_query_refinement": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "query_expansion_phrases": [
          "where state is",
          "located in"
        ],
        "is_person": false
      },
      {
        "field": "type",
        "enabled": false,
        "query_expansion_phrases": [],
        "is_person": false
      }
    ]
  }
}

Request examples

edit

curl \
--request 'PUT' \
--url "https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io:443/api/ws/v1/sources/$CONTENT_SOURCE_ID" \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $ACCESS_TOKEN" \
--data '
{
  "name": "Test Update",
  "schema": {
    "title": "text",
    "url": "text",
    "my_field": "text"
  },
  "display": {
    "title_field": "title",
    "url_field": "url",
    "type_field": "my_field",
    "detail_fields": [
      {
        "field_name": "my_field",
        "label": "My Updated Field Display"
      }
    ],
    "color": "#222222"
  },
  "is_searchable": true,
  "facets": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "display_name": "State",
        "transform": "titleize"
      },
      {
        "field": "type",
        "enabled": false
      }
    ]
  },
  "automatic_query_refinement": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "query_expansion_phrases": [
          "where state is",
          "located in"
        ],
        "is_person": false
      },
      {
        "field": "type",
        "enabled": false,
        "query_expansion_phrases": [],
        "is_person": false
      }
    ]
  }
}
'

curl \
--request 'PUT' \
--url "http://localhost:3002/api/ws/v1/sources/$CONTENT_SOURCE_ID" \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $ACCESS_TOKEN" \
--data '
{
  "name": "Test Update",
  "schema": {
    "title": "text",
    "url": "text",
    "my_field": "text"
  },
  "display": {
    "title_field": "title",
    "url_field": "url",
    "type_field": "my_field",
    "detail_fields": [
      {
        "field_name": "my_field",
        "label": "My Updated Field Display"
      }
    ],
    "color": "#222222"
  },
  "is_searchable": true,
  "facets": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "display_name": "State",
        "transform": "titleize"
      },
      {
        "field": "type",
        "enabled": false
      }
    ]
  },
  "automatic_query_refinement": {
    "overrides": [
      {
        "field": "state",
        "enabled": true,
        "query_expansion_phrases": [
          "where state is",
          "located in"
        ],
        "is_person": false
      },
      {
        "field": "type",
        "enabled": false,
        "query_expansion_phrases": [],
        "is_person": false
      }
    ]
  }
}
'

Responses

edit

200 OK

The single updated content source is returned.

{ <ContentSource> }

See the above content source JSON representation.

400 Bad Request

The request body did not meet requirements. See the errors array to determine what may be wrong.

{
  "errors": [
    "The property '#/schema/my_field' value \"integer\" did not match one of the following values: text, geolocation, number, date",
    "The property '#/display' did not contain a required property of 'title_field'",
    "The property '#/is_searchable' did not match any of the following types: boolean",
    "The property '#/name' value \"&special_chars*\" did not match the regex '(?-mix:\\A[\\w\\s.()'\\-\\/]*\\z)'"
  ]
}

400 Illegal Schema Change

The request body included a schema that, while syntactically correct, would require field type changes for document values that cannot match the new type. This was not detected until after some updates may have been applied, but did prevent the schema update from being applied as expected. The response includes both the current state of the updated content source, and the errors in updating the schema.

{
  "updated" : { <ContentSource> },
  "errors": [
    "Field 'description' cannot be changed to type 'float'. Example IDs that would fail: [\"park_acadia\", \"park_badlands\", \"park_big-bend\", \"park_bryce-canyon\", \"park_canyonlands\", \"park_capitol-reef\", \"park_carlsbad-caverns\", \"park_congaree\", \"park_cuyahoga-valley\", \"park_denali\"]"
  ]
}

See the above content source JSON representation.

400 Invalid Display

The request body included a display configuration that, while syntactically correct, did not align with the content source’s schema. This was not detected until after some updates may have been applied, but did prevent the display settings from being applied as expected. The response includes both the current state of the updated content source, and the errors in configuring the display.

{
  "updated" : { <ContentSource> },
  "errors" : [
    "Invalid display setting: title_field: \"title\" is not a field on the content source",
    "Invalid display setting: subtitle_field: \"states\" is not a field on the content source",
    "Invalid display setting: description_field: \"description\" is not a field on the content source",
    "Invalid display setting: url_field: \"nps_link\" is not a field on the content source",
    "Invalid display setting: detail_fields: \"states\" is not a field on the content source, \"description\" is not a field on the content source, \"date_established\" is not a field on the content source, \"visitors\" is not a field on the content source, \"location\" is not a field on the content source, \"world_heritage_site\" is not a field on the content source, \"square_km\" is not a field on the content source, \"acres\" is not a field on the content source"

  ]
}

See the above content source JSON representation.

401 Unauthorized

The authorization header was missing from the request or authentication failed.

404 Not Found

The specified content source ID does not exist

The authenticated user is not an Admin user, and therefore does not have permission.

Modifying a content source’s schema

edit

Note that when making changes to a configurable or custom source’s schema, you can only add new schema fields or modify existing schema field types.

It is not currently possible to remove a schema field. If a document was indexed with a field that should not be searchable, you must delete or reindex the document without the field present. In some cases, it may be easier to remove and recreate the content source entirely to correct a schema.

Changing a content source’s schema field type for an existing field

edit

If you need to change the type of a schema field, you need to issue an update request for the content source with the updated schema. The easiest way to do it would be first to issue a GET request to get the whole content source body, apply the changes you want to the schema and then issue the update using the modified body. For example, if the current schema is:

{
  "schema": {
    "country": "text",
    "criteria": "text",
    "description": "text",
    "transboundary": "text",
    "title": "text",
    "type": "text",
    "url": "text",
    "year_added": "number",
    "location": "geolocation",
    "justification": "text",
    "area_hectacres": "number",
    "region": "text"
  }
}

but your field type changed and now you want year_added to also be represented as text, you would simply replace it with:

  "schema": {
    "country": "text",
    "criteria": "text",
    "description": "text",
    "transboundary": "text",
    "title": "text",
    "type": "text",
    "url": "text",
    "year_added": "text",
    "location": "geolocation",
    "justification": "text",
    "area_hectacres": "number",
    "region": "text"
  }

Adding a content source’s schema field

edit

If you need to add a new schema field, you need to issue an update request for the content source with the updated schema. The easiest way to do it would be first to issue a GET request to get the whole content source body, apply the changes you want to the schema and then issue the update using the modified body. For example, if the current schema is:

  "schema": {
    "country": "text",
    "criteria": "text",
    "description": "text",
    "transboundary": "text",
    "title": "text",
    "type": "text",
    "url": "text",
    "year_added": "number",
    "location": "geolocation",
    "justification": "text",
    "area_hectacres": "number",
    "region": "text"
  }

and you’re adding a new field of type date, you would just add it to the end of the list like this:

  "schema": {
    "country": "text",
    "criteria": "text",
    "description": "text",
    "transboundary": "text",
    "title": "text",
    "type": "text",
    "url": "text",
    "year_added": "number",
    "location": "geolocation",
    "justification": "text",
    "area_hectacres": "number",
    "region": "text",
    "system_modified_date": "date"
  }

Upload content source icons

edit

Upload icons for a single custom content source.

Only custom content sources can be updated with this endpoint. Other service types cannot be updated programmatically.

PUT /api/ws/v1/sources/<content-source-id>/icon

Request headers

edit

Authorization

Bearer token permitting API access.

For more authentication methods see API Authentication Reference

Content-Type

application/json. This endpoint accepts only JSON PUT data.

Request parameters

edit

content-source-id (required): The string ID for the content source to be updated.

Request body

edit

Content source icon definition (required)

The JSON object defining the icon of the content source.

main_icon (optional)

Base64 encoded string of the main icon content, or null to delete the main_icon.

The main_icon will be displayed in Search Result Card. Default icon will be displayed if the main_icon is not provided.

The main_icon is required if there’s no main_icon uploaded for the content source.

The main_icon must not be over 1 MB in size.

The main_icon must be in the format of PNG.

alt_icon (optional)

Base64 encoded string of the alt icon content, or null to delete the alt_icon.

The alt_icon will be displayed in all places except Search Result Card. The main_icon will be displayed if the alt_icon is not provided. Default icon will be displayed if neither main_icon and alt_icon is provided.

The alt_icon must not be over 1 MB in size.

The alt_icon must be in the format of PNG.

Example

edit

Below is an example content source icon definition’s JSON representation:

{
  "main_icon": "Base64 encoded string",
  "alt_icon": "Base64 encoded string"
}

Request examples

edit

curl \
--request 'PUT' \
--url "https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io:443/api/ws/v1/sources/$CONTENT_SOURCE_ID/icon" \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $ACCESS_TOKEN" \
--data '
{
  "main_icon": "Base64 encoded string",
  "alt_icon": "Base64 encoded string"
}
'

curl \
--request 'PUT' \
--url "http://localhost:3002/api/ws/v1/sources/$CONTENT_SOURCE_ID/icon" \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $ACCESS_TOKEN" \
--data '
{
  "main_icon": "Base64 encoded string",
  "alt_icon": "Base64 encoded string"
}
'

Responses

edit

200 OK

An acknowledgement response is returned.

{
  "results": {
    "main_icon": "success",
    "alt_icon": "success"
  }
}

400 Non-custom content source

The content source is not a custom content source.

{
  "errors": [
    "Invalid parameter(s): Content source $CONTENT_SOURCE_ID is not a custom API source"
  ]
}

400 No icon is provided

No icon is provided.

{
  "errors": [
    "At least one icon needs to be provided"
  ]
}

400 Alt icon is uploaded without main icon

Alt icon exists without main icon.

{
  "errors": [
    "Alt icon can't be uploaded without main icon"
  ]
}

400 Unsupported file type

The file type of the icon is not supported.

{
  "errors": [
    "Main icon must be in the format of image/png",
    "Alt icon must be in the format of image/png"
  ]
}

400 Uploaded icon size is over limit

The uploaded icon is over the maximum size allowed.

{
  "errors": [
    "Main icon exceeds the max size allowed: 1 MB",
    "Alt icon exceeds the max size allowed: 1 MB"
  ]
}

401 Unauthorized

The authorization header was missing from the request or authentication failed.

404 Not Found

The specified content source ID does not exist

The authenticated user is not an Admin user, and therefore does not have permission.

Remove a content source

edit

Remove a single content source by its ID.

Only organization content sources can be deleted with this endpoint. Private content sources cannot be deleted with this endpoint.

DELETE /api/ws/v1/sources/<content-source-id>

Request headers

edit

Authorization

Bearer token permitting API access.

For more authentication methods see API Authentication Reference

Request parameters

edit

content-source-id (required): The string ID for the content source to be removed.

Request examples

edit

curl \
--request 'DELETE' \
--url "https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io:443/api/ws/v1/sources/$CONTENT_SOURCE_ID" \
--header "Authorization: Bearer $ACCESS_TOKEN"

curl \
--request 'DELETE' \
--url "http://localhost:3002/api/ws/v1/sources/$CONTENT_SOURCE_ID" \
--header "Authorization: Bearer $ACCESS_TOKEN"

Responses

edit

200 OK

An acknowledgement that the content source was deleted

{ "deleted" : true }

400 Bad Request

The provided content_source_id is for a private content source, which is not allowed.

{ "errors": ["Invalid parameter(s): Private content sources cannot be deleted by this API"] }

401 Unauthorized

The authorization header was missing from the request or authentication failed.

404 Not Found

The authenticated user is not an Admin user, and therefore does not have permission.

The provided content_source_id does not exist.

Get a content source’s automatic query refinement details

edit

Where the contents of Content source JSON representation contain the user-defined overrides for automatic query refinements, this API also includes the defaults for the content source, and the explicit results that can be logically derived from applying the overrides to the defaults.

GET /api/ws/v1/sources/<content-source-id>/automatic_query_refinement

Request headers

edit

Authorization

Bearer token permitting API access.

For more authentication methods see API Authentication Reference

Request parameters

edit

content-source-id (required): The string ID for the content source that contains the desired document.

Request examples

edit

curl \
--request 'GET' \
--url "https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io:443/api/ws/v1/sources/$CONTENT_SOURCE_ID/automatic_query_refinement" \
--header "Authorization: Bearer $ACCESS_TOKEN"

curl \
--request 'GET' \
--url "http://localhost:3002/api/ws/v1/sources/$CONTENT_SOURCE_ID/automatic_query_refinement" \
--header "Authorization: Bearer $ACCESS_TOKEN"

Responses

edit

200 OK

The single content source’s automatic query refinement details are returned.

This includes the defaults for the content source, the user-provided overrides, and the results of logically combining the defaults with the overrides

{
  "overrides": [
    {
      "field": "one",
      "enabled": true,
      "query_expansion_phrases": [
        "one is"
      ],
      "is_person": false
    },
    {
      "field": "two",
      "enabled": true,
      "query_expansion_phrases": [
        "two is"
      ],
      "is_person": false
    }
  ],
  "defaults": [
    {
      "field": "assigned_to",
      "query_expansion_phrases": [
        "assigned to"
      ],
      "is_person": true
    },
    {
      "field": "created_by",
      "query_expansion_phrases": [
        "creator is",
        "created by",
        "edited by",
        "modified by"
      ],
      "is_person": true
    },
    {
      "field": "created_by_email",
      "query_expansion_phrases": [
        "creator is",
        "created by",
        "edited by",
        "modified by"
      ],
      "is_person": true
    },
    {
      "field": "shared_by",
      "query_expansion_phrases": [
        "from",
        "shared by"
      ],
      "is_person": true
    },
    {
      "field": "shared_by_email",
      "query_expansion_phrases": [
        "from",
        "shared by"
      ],
      "is_person": true
    },
    {
      "field": "updated_by",
      "query_expansion_phrases": [
        "edited by",
        "updated by",
        "modified by"
      ],
      "is_person": true
    },
    {
      "field": "updated_by_email",
      "query_expansion_phrases": [
        "edited by",
        "updated by",
        "modified by"
      ],
      "is_person": true
    },
    {
      "field": "extension",
      "query_expansion_phrases": [],
      "is_person": false
    },
    {
      "field": "status",
      "query_expansion_phrases": [
        "with status",
        "status is",
        "in state",
        ""
      ],
      "is_person": false
    },
    {
      "field": "state",
      "query_expansion_phrases": [
        "with state",
        "state is",
        "in state",
        ""
      ],
      "is_person": false
    },
    {
      "field": "tags",
      "query_expansion_phrases": [
        "with tag",
        "tagged",
        "tagged with"
      ],
      "is_person": false
    },
    {
      "field": "type",
      "query_expansion_phrases": [],
      "is_person": false
    },
    {
      "field": "impact",
      "query_expansion_phrases": [
        "with impact",
        "impact is"
      ],
      "is_person": false
    },
    {
      "field": "urgency",
      "query_expansion_phrases": [
        "with urgency",
        "urgency is"
      ],
      "is_person": false
    },
    {
      "field": "priority",
      "query_expansion_phrases": [
        "with priority",
        "priority is"
      ],
      "is_person": false
    },
    {
      "field": "office_location",
      "query_expansion_phrases": [
        "in",
        "from",
        "located in"
      ],
      "is_person": false
    }
  ],
  "results": [
    {
      "field": "assigned_to",
      "query_expansion_phrases": [
        "assigned to"
      ],
      "is_person": true
    },
    {
      "field": "created_by",
      "query_expansion_phrases": [
        "creator is",
        "created by",
        "edited by",
        "modified by"
      ],
      "is_person": true
    },
    {
      "field": "created_by_email",
      "query_expansion_phrases": [
        "creator is",
        "created by",
        "edited by",
        "modified by"
      ],
      "is_person": true
    },
    {
      "field": "shared_by",
      "query_expansion_phrases": [
        "from",
        "shared by"
      ],
      "is_person": true
    },
    {
      "field": "shared_by_email",
      "query_expansion_phrases": [
        "from",
        "shared by"
      ],
      "is_person": true
    },
    {
      "field": "updated_by",
      "query_expansion_phrases": [
        "edited by",
        "updated by",
        "modified by"
      ],
      "is_person": true
    },
    {
      "field": "updated_by_email",
      "query_expansion_phrases": [
        "edited by",
        "updated by",
        "modified by"
      ],
      "is_person": true
    },
    {
      "field": "extension",
      "query_expansion_phrases": [],
      "is_person": false
    },
    {
      "field": "status",
      "query_expansion_phrases": [
        "with status",
        "status is",
        "in state",
        ""
      ],
      "is_person": false
    },
    {
      "field": "state",
      "query_expansion_phrases": [
        "with state",
        "state is",
        "in state",
        ""
      ],
      "is_person": false
    },
    {
      "field": "tags",
      "query_expansion_phrases": [
        "with tag",
        "tagged",
        "tagged with"
      ],
      "is_person": false
    },
    {
      "field": "type",
      "query_expansion_phrases": [],
      "is_person": false
    },
    {
      "field": "impact",
      "query_expansion_phrases": [
        "with impact",
        "impact is"
      ],
      "is_person": false
    },
    {
      "field": "urgency",
      "query_expansion_phrases": [
        "with urgency",
        "urgency is"
      ],
      "is_person": false
    },
    {
      "field": "priority",
      "query_expansion_phrases": [
        "with priority",
        "priority is"
      ],
      "is_person": false
    },
    {
      "field": "office_location",
      "query_expansion_phrases": [
        "in",
        "from",
        "located in"
      ],
      "is_person": false
    },
    {
      "field": "one",
      "query_expansion_phrases": [
        "one is"
      ],
      "is_person": false
    },
    {
      "field": "two",
      "query_expansion_phrases": [
        "two is"
      ],
      "is_person": false
    }
  ]
}

401 Unauthorized

The authorization header was missing from the request or authentication failed.

404 Not Found

The authenticated user is not an Admin user, and therefore does not have permission.

The authenticated user has specified a private content source that is not theirs, and they therefore do not have permission.

The provided content_source_id does not exist.

Get a document

edit

Retrieve a single content source document, given the IDs of both the content source and the document.

GET /api/ws/v1/sources/<content-source-id>/documents/<document-id>

Request headers

edit

Authorization

Bearer token permitting API access.

For more authentication methods see API Authentication Reference

Request parameters

edit

content-source-id (required): The string ID for the content source that contains the desired document.
document-id (required): The string ID for the document to be retrieved.

Request examples

edit

curl \
--request 'GET' \
--url "https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io:443/api/ws/v1/sources/$CONTENT_SOURCE_ID/documents/$DOCUMENT_ID" \
--header "Authorization: Bearer $ACCESS_TOKEN"

curl \
--request 'GET' \
--url "http://localhost:3002/api/ws/v1/sources/$CONTENT_SOURCE_ID/documents/$DOCUMENT_ID" \
--header "Authorization: Bearer $ACCESS_TOKEN"

Responses

edit

200 OK

The single content source document is returned.

The only guaranteed response fields are id, source, content_source_id, and last_updated. The others are content source specific, and must be part of the content source’s schema. You can view a content source’s schema with the get content source API.

{
  "id":  "some_string_id",
  "source": "custom",
  "content_source_id": "603d2c8319cbdbc907675c02",
  "last_updated": "2021-03-02T22:41:16+00:00",
  "some_text_field": "some text value",
  "some_number_field": 42,
  "some_date_field": "2021-03-02T22:41:16+00:00",
  "some_geolocation_field": "44.35,-68.21",
  "_allow_permissions": ["perm1", "perm2"],
  "_deny_permissions": ["perm3", "perm4"]
}

401 Unauthorized

The authorization header was missing from the request or authentication failed.

404 Not Found

The authenticated user is not an Admin user, and therefore does not have permission.

The authenticated user has specified a private content source that is not theirs, and they therefore do not have permission.

The specified document has document-level-permissions which prohibit access to the authenticated user.

The provided content_source_id does not exist.

The provided document_id does not exist.

Manage content source synchronization

edit

Manage the synchronization state of the content source - either by starting a new synchronization job or by interrupting an existing one with per job-type granularity.

POST /api/ws/v1/sources/<content-source-id>/sync/jobs

Request headers

edit

Authorization

Bearer token permitting API access.

For more authentication methods see API Authentication Reference

Request parameters

edit

content-source-id (required): The string ID for the content source to operate on.

Query Parameters

edit

job-type (optional): Array of strings that contains job types to run the command for. Allowed values are: full, incremental, delete, permissions. If this value is omitted, all job types will be considered.

Request body

edit

command (required)

The string start or the string interrupt. The value specifies the type of job command to run.

Submit a start command to start a synchronization job or jobs for a content source according to the job-type query parameter. If some of the jobs are already running, an error will be returned. To restart these jobs instead of receiving an error, see force_interrupt flag.

Submit an interrupt command to interrupt a running synchronization job or jobs according to the job-type query parameter. No error is produced if no job of the specified type is running.

force_interrupt (optional)

The boolean that can be provided if the start command is sent. force_interrupt: true will indicate that the system needs to interrupt jobs of specified types before starting new ones.

Request examples

edit

Start synchronization job request template:

curl \
--request 'POST' \
--url '<WORKPLACE_SEARCH_BASE_URL>/api/ws/v1/sources/<content-source-id>/sync/jobs' \
--header 'Authorization: Bearer <ACCESS_TOKEN>' \
--header 'Content-Type: application/json' \
--data '{
    "command": "start",
    "force_interrupt": false
}'

Interrupt synchronization job template:

curl \
--request 'POST' \
--url '<WORKPLACE_SEARCH_BASE_URL>/api/ws/v1/sources/<content-source-id>/sync/jobs' \
--header 'Authorization: Bearer <ACCESS_TOKEN>' \
--header 'Content-Type: application/json' \
--data '{
    "command": "interrupt"
}'

Responses

edit

200 OK

A results object is returned that contains information about the affected jobs.

When a start command is issued, an array of affected jobs is contained in a started field. When an interrupt command is issued, an array of affected jobs is contained in an interrupted field. Jobs are guaranteed to have the following fields: id, job_type, status, created_at and last_updated_at.

{
    "results": {
        "started": [
            {
                "id": "6103d19a5917cd61ab13665f",
                "job_type": "full",
                "status": "enqueued",
                "created_at": "2021-07-30T10:16:58+00:00",
                "last_updated_at": "2021-07-30T10:16:58+00:00"
            },
            {
                "id": "6103d19b5917cd61ab136660",
                "job_type": "incremental",
                "status": "enqueued",
                "created_at": "2021-07-30T10:16:59+00:00",
                "last_updated_at": "2021-07-30T10:16:59+00:00"
            },
            {
                "id": "6103d19b5917cd61ab136661",
                "job_type": "delete",
                "status": "enqueued",
                "created_at": "2021-07-30T10:16:59+00:00",
                "last_updated_at": "2021-07-30T10:16:59+00:00"
            },
            {
                "id": "6103d19b5917cd61ab136662",
                "job_type": "permissions",
                "status": "enqueued",
                "created_at": "2021-07-30T10:16:59+00:00",
                "last_updated_at": "2021-07-30T10:16:59+00:00"
            }
        ]
    }
}

400 Bad Request

The request body did not meet requirements. See the errors array to determine what may be wrong.

401 Unauthorized

The authorization header was missing from the request, or authentication failed.

404 Not Found

The authenticated user is not an Admin user, and therefore does not have permission.

The authenticated user has specified a private content source that is not theirs, and they therefore do not have permission.

The provided content_source_id does not exist.

« API Authentication Reference Custom sources indexing API reference »