Add a document to a corpus

POST 
/v2/corpora/:corpus_key/documents

Add a document to a corpus. This endpoint supports two document formats, structured and core.

• Structured documents have a more conventional structure that provide document sections and parts in a format created by Vectara's proprietary strategy automatically. You provide a logical document structure, and Vectara handles the partitioning.
• Core documents differ in that they follow an advanced, granular structure that explicitly defines each document part in an array. Each part becomes a distinct, searchable item in query results. You have precise control over the document structure and content.

For more details, see Indexing.

Request

Path Parameters

corpus_key CorpusKeyrequired

Possible values: <= 50 characters, Value must match regular expression [a-zA-Z0-9_\=\-]+$

The unique key identifying the queried corpus.

Header Parameters

Request-Timeout integer

Possible values: >= 1

The API will make a best effort to complete the request in the specified seconds or time out.

Request-Timeout-Millis integer

Possible values: >= 1

The API will make a best effort to complete the request in the specified milliseconds or time out.

application/json

Body

oneOf

CoreDocument

id stringrequired

The document ID must be unique within the corpus.

type stringrequired

Default value: core

When the type of the indexed document is core the rest of the object is expected to follow this schema. This schema allows precise specification of document chunks that get directly translated to retrieve search results.

metadata object

Arbitrary object of document level metadata. Properties of this object can be used by document filters if defined as a corpus filter attribute.

property name* any

Arbitrary object of document level metadata. Properties of this object can be used by document filters if defined as a corpus filter attribute.

tables object[]

The tables that this document contains.

Array [

id string

The unique ID of the table within the document.

title string

The title of the table.

data object

The data of the table.

headers array[]

The headers of the table.

rows array[]

The rows in the data.

description string

The description of the table.

]

document_parts object[]required

Possible values: >= 1

Parts of the document that make up the document.

Array [

text stringrequired

The text of the document part.

metadata object

The metadata for a document part. These may be used in metadata filters at query time if filter attributes are configured on the corpus.

property name* any

The metadata for a document part. These may be used in metadata filters at query time if filter attributes are configured on the corpus.

table_id string

The ID of the table that this document part belongs to.

context string

The context text for the document part.

custom_dimensions object

The custom dimensions as additional weights.

property name* double

]

StructuredDocument

id stringrequired

The document ID must be unique within the corpus.

type stringrequired

Default value: structured

When the type of the indexed document is structured the rest of the object is expected to follow this schema. It allows you to create a document that follows normal document conventions. The Vectara platform will then create document parts using its internal algorithm.

title string

The title of the document.

description string

The description of the document.

metadata object

The metadata for a document as an arbitrary JSON object. Properties of this object can be used by document level filter attributes.

property name* any

The metadata for a document as an arbitrary JSON object. Properties of this object can be used by document level filter attributes.

custom_dimensions object

The custom dimensions as additional weights.

property name* double

sections object[]required

Possible values: >= 1

The subsection of the document.

Array [

id int32

The section ID. This gets converted to a metadata field automatically.

title string

The section title.

text stringrequired

The text of the section.

metadata object

Arbitrary object that becomes document part level metadata on any document part created by this section. Properties of this object can be used by document part level filters if defined as a corpus filter attribute.

property name* any

Arbitrary object that becomes document part level metadata on any document part created by this section. Properties of this object can be used by document part level filters if defined as a corpus filter attribute.

tables object[]

The tables that this section contains.

Array [

id string

The unique ID of the table within the document.

title string

The title of the table.

data object

The data of the table.

headers array[]

The headers of the table.

rows array[]

The rows in the data.

description string

The description of the table.

]

sections undefined[]

The sections that this section contains.

]

chunking_strategy object

(Optional) Choose how to split documents into chunks during indexing. If you do not set a chunking strategy, the platform uses the default strategy which creates one chunk (docpart) per sentence.

oneOf

MaxCharsChunkingStrategy

type string

Default value: max_chars_chunking_strategy

When setting the type to max_chars_chunking_strategy, you can control the size of chunks (docparts).

max_chars_per_chunk int32required

Possible values: >= 100

Specifies the maximum number of characters per chunk.

The platform adds sentences to a chunk until the total number of characters exceeds the limit.

If a single sentence exceeds the limit, it splits the sentence across chunks. Note: This is the only case where the chunk may not contain a complete sentence.

Responses

201

Document added to the corpus.

application/json

Schema

Schema

id string

The document ID.

metadata object

The document metadata.

property name* any

The document metadata.

tables object[]
The tables that this document contains. Tables are not available when table extraction is not enabled.
Array [
id string
The unique ID of the table within the document.
title string
The title of the table.
data object
The data of the table.
headers array[]
The headers of the table.
rows array[]
The rows in the data.
description string
The description of the table.
]
parts object[]
Parts of the document that make up the document. However, parts are not available when retrieving a list of documents or when creating a document. This property is only available when retrieving a document by ID.
Array [
text stringrequired
The text of the document part.
metadata object
The metadata for a document part. These may be used in metadata filters at query time if filter attributes are configured on the corpus.
property name* any
The metadata for a document part. These may be used in metadata filters at query time if filter attributes are configured on the corpus.
context string
The context text for the document part.
custom_dimensions object
The custom dimensions as additional weights.
property name* double
]
storage_usage object

How much storage the document used. This information is currently not returned when retrieving the document, and only returned when indexing a document.

bytes_used int64

Number of bytes used by document counting towards maximum corpus size, and towards any billing plans.

metadata_bytes_used int64

Number of metadata bytes used by a document.

Example (from schema)

{
  "id": "my-doc-id",
  "metadata": {},
  "tables": [
    {
      "id": "table_1",
      "title": "string",
      "data": {
        "headers": [
          [
            {
              "text_value": "string",
              "int_value": 0,
              "float_value": 0,
              "bool_value": true,
              "colspan": 0,
              "rowspan": 0
            }
          ]
        ],
        "rows": [
          [
            {
              "text_value": "string",
              "int_value": 0,
              "float_value": 0,
              "bool_value": true,
              "colspan": 0,
              "rowspan": 0
            }
          ]
        ]
      },
      "description": "string"
    }
  ],
  "parts": [
    {
      "text": "I'm a nice document part.",
      "metadata": {
        "nice_rank": 9000
      },
      "context": "string",
      "custom_dimensions": {}
    }
  ],
  "storage_usage": {
    "bytes_used": 0,
    "metadata_bytes_used": 0
  }
}

400

Document creation request was malformed.

application/json

Schema

Schema

field_errors object
The errors that relate to specific fields in the request.
property name* string
messages message[]
request_id string

The ID of the request that can be used to help Vectara support debug what went wrong.

Example (from schema)

{
  "field_errors": {},
  "messages": [
    "string"
  ],
  "request_id": "string"
}

403

Permissions do not allow adding a document to the corpus.

application/json

Schema

Schema

messages message[]

The messages describing why the error occurred.

request_id string

The ID of the request that can be used to help Vectara support debug what went wrong.

Example (from schema)

{
  "messages": [
    "Internal server error."
  ],
  "request_id": "string"
}

404

Corpus not found.

application/json

Schema

Schema

id string

The ID cannot be found.

messages message[]
request_id string

ID of the request that can be used to help Vectara support debug what went wrong.

Example (from schema)

{
  "id": "string",
  "messages": [
    "string"
  ],
  "request_id": "string"
}

Loading...