Sources
A source is where a pipeline reads records from. Each record is fed to a new agent
session for processing. The source object on a pipeline is discriminated by
its type, which selects the connector and its configuration.
Available sources
Amazon S3
s3Ingest objects from an Amazon S3 bucket or any S3-compatible service.
Google Drive
google_driveIngest files from Google Drive shared drives and My Drive folders.
SharePoint
sharepointIngest files from a SharePoint site using Azure AD app-only credentials.
Box
boxIngest files from a Box enterprise, inheriting collaborations as document ACL.
Wolken
wolken_kbIngest knowledge-base articles from a Wolken ServiceDesk instance.
Web
webIngest pages from a website by sitemap, crawl, or both.
Confluence
confluenceIngest pages from Confluence Cloud or Data Center, inheriting read restrictions as ACL.
Fluid Topics pipeline source
The Fluid Topics pipeline source ingests content from a Fluid Topics tenant through the standard Knowledge Hub REST API and sends each record to the pipeline's transform agent for indexing.
Capabilities
Every source supports incremental sync and resolves source metadata at fetch time. They differ in which metadata buckets they populate and how they authenticate.
| Source | type | ACL metadata | Authentication |
|---|---|---|---|
| Amazon S3 | s3 | Object ACL (owners, editors, readers, public/authenticated groups) | Access key ID and secret |
| Google Drive | google_drive | Drive permissions | OAuth service account |
| SharePoint | sharepoint | None | App registration (Microsoft Graph) |
| Box | box | Collaborations and shared-link access | Client Credentials Grant |
| Wolken | wolken_kb | Org-wide read (audience-level) | Wolken data API credentials |
| Web | web | None | Optional per-request credential |
| Confluence | confluence | Page read restrictions and space audience | RemoteAuth (Cloud or Data Center) |
Deletes are not propagated by any source. A record the connector can no longer retrieve stops appearing in new runs. The pipeline emits no delete signal, so anything a previous run already produced downstream, for example, a document indexed into a corpus, is left in place.