Custom sources: Search content from anywhere with Elastic Workplace Search
Elastic Workplace Search is rooted in a simple, but powerful mission: bringing information, content, and files from every corner of your organization into a singular, centralized source of truth. Thanks to its portfolio of first-party integrations, Workplace Search easily connects to a variety of popular content platforms. Whether your teams rely on images from Dropbox, accounts from Salesforce, tickets from Zendesk, or sites from SharePoint Online for their daily tasks, you can synchronize the goldmine of information across all of your collaboration tools with just a few clicks. In a matter of minutes, you can create a relevant search experience that surfaces the right content to the right individual, dramatically improving work efficiency and accuracy at scale.
Such a straightforward way to connect widely adopted data sources is fantastic, but what if your team also relies on homegrown applications critical to your unique organizational knowledge landscape? What about niche platforms or legacy content sources? Here again, there’s a simple answer: custom sources.
The anatomy of a custom source
Custom sources operate much like any other data source connected to the Workplace Search experience. Once created, a custom source becomes a unique repository of content with its own data structure and configurations. It can be updated in real-time, and features such as content filtering and automatic keyword detection at query time are enabled with no additional configuration required.
To create a custom source, head to the Sources area of the Workplace Search dashboard, click Add an organization content source and locate Create a Custom API Source:
From here, you must name the source to be created. Do so wisely: It will be used to identify the content source in the search experiences used by the various teams in your organization. Hit that green button and voilà! You’ve created a new container for the content you’re looking to expose from the search experience:
Giving it a push
We now have a content platform filled with valuable information on one side, and a custom source ready to receive the data on the other. The next step involves building a small program that bridges the gap between the two repositories, commonly referred to as a connector. While precise instructions on how to build a connector fall outside the scope of this piece, there a few guiding principles to keep in mind as you are developing synchronization software:
- Standalone vs. embedded: You may choose to develop a small synchronization script that lives on its own, or embed the Workplace Search ingestion logic directly within the application for which you want to make the information searchable.
- Data modeling and transformation: While the content source may have large, complex data structures, astutely choosing which attributes to index will lead to a more relevant search experience.
- Document-level permissions: Most content platforms are backed by authorization systems that allow certain pieces of information to be accessed by certain individuals, but not others. Using document permissions, you can maintain the same level of access at the search experience level.
- Real-time vs. scheduled: Custom sources can be updated in real-time via the API, but some sources may benefit from a scheduled ingestion strategy, such as once a day.
- Observability: Keeping track of connector activity may help you understand any changes that would cause the indexing process to behave differently over time.
For more information on the Custom Sources API and its features, refer to the exhaustive guide found in the Elastic Workplace Search documentation.
Searching with a cause
With content now freely flowing into Workplace Search, your teammates can now search over its content alongside the other sources connected to the platform. Custom sources are a first-class citizen of the search experience, the same way the Google Drive or Salesforce integrations are. This translates into a seamless experience when searching across content repositories. For example, searching for acme inc contract updated last week will return results both from Dropbox and our bespoke Legal Document Vault custom source, automatically filtered down to the information updated in the last 7 days:
Tailored relevance for all teams
Custom sources can be assigned to Groups, ensuring that the right individuals and teams have access to the right content at the right time. Some teams may even rely more heavily on information found in a custom source, which makes for a wonderful use of the Source Prioritization tool. Source Prioritization applies relative algorithmic importance to the information matching in the data repository:
Zen and the art of custom source maintenance
The great news about custom sources is that you are in control at any point in time. Should the data structure evolve over time, you can update the underlying schema directly from the administrative dashboard, with no downtime or degraded search experience:
There is also a reasonable chance that you may want to update the way results are presented to your teammates over time. Workplace Search offers an intuitive interface to properly assign the right pieces of data to the result layout, used across the various search clients provided with the platform.
