StreamQuery

POST /v1/stream-query

Stream relevant results, highlight relevant snippets, and do Retrieval Augmented Generation (RAG).

Some tips for this API:

This operation authenticates with either an API Key or OAuth 2.0 (in a JWT "Bearer Token"). You can find details of how to set up an API key or use OAuth 2.0 here.
By default, Vectara only uses its neural/semantic retrieval model, and does not attempt to use keyword matching. To enable hybrid search with a mix of both keyword and neural results, edit the lambda value. Additional documentation on hybrid search is here.
Vectara can automatically provide a generative summary with the query results for Retrieval Augmented Generation (RAG). Additional details on RAG in Vectara can be found here.
The Stream Query API takes an array of queries, since you can have Vectara perform more than 1 query in a single API call. The results that are returned are thus in an array as well, with each element in the response array corresponding to the input query array.

Request

Header Parameters

customer-id integerrequired

Enter the Customer ID to use for the request.

timeout string

Default value: 30S

(Optional) Enter the timeout value of the request in seconds, such as 10S or 30S.

application/json

Body

required

query object[]

Array [

query string

The query text to use from the end user.

start int64

numResults int64required

Possible values: >= 1

The number of results to return.

contextConfig object

charsBefore int32

chars_before is used for showing the end user the characters leading up to the result snippet. This can help the end-user understand the context of that result. Ignored if sentences_before is set.

charsAfter int32

chars_after is used for showing the end user the characters after the result snippet. This can help the end-user understand the context of that result. Ignored if sentences_before is set.

sentencesBefore int32required

sentences_before is used for showing the end user the sentences leading up to the result snippet. This can help the end-user understand the context of that result.

sentencesAfter int32required

sentences_after is used for showing the end user the sentences leading up to the result snippet. This can help the end-user understand the context of that result.

startTag stringrequired

The tag that wraps the snippet at the start. Often this is used to provide a start HTML/XML tag or some other delimiter you can use in an application to understand where to provide highlighting in your UI and understand where the context before ends and the snippet begins.

endTag stringrequired

The tag that wraps the snippet at the end. Often this is used to provide a start HTML/XML tag or some other delimiter you can use in an application to understand where to provide highlighting in your UI and understand where the snippet ends and the context after begins.

corpusKey object[]required

The query is run on all these corpora, and the results are merged together in the response, ranked by score.

Array [

customerId int64required

The Customer ID.

corpusId int64required

Possible values: >= 1

The Corpus ID.

semantics CorpusKeySemantics

Possible values: [DEFAULT, QUERY, RESPONSE]

Default value: DEFAULT

Semantics controls the interpretation of the query string by the server, and can be used to override the default semantics assigned in the corpus definition.

DEFAULT: Use corpus-assigned semantics. This is the most common setting.
QUERY: Use query semantics. This is also common.
RESPONSE: Use response semantics. Usage of this is rare.

metadataFilter string

lexicalInterpolationConfig object

Configuration for a linear interpolation.

lambda float

dim object[]

Array [

name stringrequired

weight numberrequired

]

summary object[]

Optionally, one or more requests to summarize the results.

Array [

summarizerPromptName string

The name of the summarizer+prompt combination to use for summarization.

maxSummarizedResults int64required

Maximum number of results to summarize.

responseLang stringrequired

ISO 639-1 or ISO 639-3 language code for the response, or "auto" to indicate that the auto-detected language of the incoming query should be used.

promptText string

Vectara manages both system and user roles and prompts for the generative LLM out of the box by default. However, Scale customers can override the prompt_text via this variable. The prompt_text is in the form of an Apache Velocity template. For more details on how to configure the prompt_text, see the long-form documentation at https://docs.vectara.com/docs/prompts/vectara-prompt-engine See https://vectara.com/pricing/ for more details on becoming a Scale customer.

debug boolean

Debugging the generative prompt is currently a Scale-only feature. See https://vectara.com/pricing/ for more details on becoming a Scale customer.

responseChars int64

Controls the length of the summary. This is a rough estimate and not a hard limit: the end summary can be longer or shorter than this value. This is currently a Scale-only feature. See https://vectara.com/pricing/ for more details on becoming a Scale customer.

modelParams object

Parameters for the summarizer model. These are currently a Scale-only feature. See https://vectara.com/pricing/ for more details on becoming a Scale customer. WARNING: This is an experimental feature, and breakable at any point with virtually no notice. It is meant for experimentation to converge on optimal parameters that can then be set in the prompt definitions.

maxTokens int64

temperature float

The sampling temperature to use. Higher values make the summary more random, while lower values make it more focused and deterministic.

frequencyPenalty float

Higher values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

presencePenalty float

Higher values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

chat object

The chat request.

store boolean

Whether to store the query/answer pair.

conversationId string

The conversation id of the chat. If empty, a new conversation will be started.

factualConsistencyScore boolean

If unset or true, the response will include the factual consistency score.

]

Responses

200
default

A successful response.(streaming responses)

application/json

Schema
Example (from schema)

Schema

result object

This message contains individual partial results that are returned asynchronously by the streaming interface.

batchQueryResponse object

responseSet object[]

The response sets for queries within the batch. If using synchronous APIs for querying, the response set will be included directly in the response. However, if using the streaming APIs for query, the response set messages only set the future_id field. Later, as response sets are computed and returned over the stream, the future_id within the summary can be used for correlation.

Array [

response object[]

Array [

text string

score float

The score used for ranking results. The higher the score, the better the match.

metadata object[]

Array [

name string

value string

]

documentIndex int64

Use this ID to find the document in the ResponseSet.

corpusKey object

customerId int64required

The Customer ID.

corpusId int64required

Possible values: >= 1

The Corpus ID.

semantics CorpusKeySemantics

Possible values: [DEFAULT, QUERY, RESPONSE]

Default value: DEFAULT

Semantics controls the interpretation of the query string by the server, and can be used to override the default semantics assigned in the corpus definition.

DEFAULT: Use corpus-assigned semantics. This is the most common setting.
QUERY: Use query semantics. This is also common.
RESPONSE: Use response semantics. Usage of this is rare.

metadataFilter string

lexicalInterpolationConfig object

Configuration for a linear interpolation.

lambda float

dim object[]

Array [

name stringrequired

weight numberrequired

]

status object[]

Potentially multiple warnings.

Array [

code vectaraStatusCode

Possible values: [OK, FAILURE, UNKNOWN, INVALID_ARGUMENT, DEADLINE_EXCEEDED, ALREADY_EXISTS, PERMISSION_DENIED, RESOURCE_EXHAUSTED, FAILED_PRECONDITION, ABORTED, OUT_OF_RANGE, UNIMPLEMENTED, INTERNAL, UNAVAILABLE, DATA_LOSS, UNAUTHENTICATED, BAD_REQUEST, UNAUTHORIZED, FORBIDDEN, NOT_FOUND, METHOD_NOT_ALLOWED, CONFLICT, UNSUPPORTED_MEDIA_TYPE, TOO_MANY_REQUESTS, INTERNAL_SERVER_ERROR, NOT_IMPLEMENTED, SERVICE_UNAVAILABLE, INSUFFICIENT_STORAGE, UNPARSEABLE_RESPONSE, DISABLED_CUSTOMER, INVALID_CUSTOMER_ID, DISABLED_CORPUS, INVALID_CORPUS_ID, DISABLED_API_KEY, EXPIRED_API_KEY, INVALID_API_KEY, CMK_INACCESSIBLE, QRY__DISABLED_CORPUS, QRY__DOCUMENT_DB_FAILURE, QRY__ENCODER_FAILURE, QRY__INTERRUPTED, QRY__INVALID_CORPUS, QRY__INVALID_START, QRY__INVALID_NUM_RESULTS, QRY__INVALID_CONTEXT, QRY__MISSING_QUERY, QRY__MISSING_CORPUS, QRY__TIMEOUT, QRY__TOO_MANY_CORPORA, QRY__TOO_MANY_QUERIES, QRY__VECTOR_INDEX_FAILURE, QRY__INVALID_DIMENSION, QRY__INVALID_CLIENTKEY, QRY__DECRYPTION_FAILURE, QRY__INVALID_RERANKER, QRY__PARTIAL_RERANK, QRY__RERANK_FAILURE, QRY__TOO_MANY_RESULT_ROWS, QRY__PARTIAL_RETRIEVAL, QRY__SMRY__INVALID_SUMMARIZER_PROMPT, QRY__SMRY__INVALID_SUMMARY_LANG, QRY__SMRY__UNSUPPORTED_SUMMARY_LANG, QRY__SMRY__PARTIAL_SUMMARY, QRY__SMRY__NO_QUERY_RESULTS, QRY__SMRY__EVAL_UNSUPPORTED_LANG, QRY__SMRY__EVAL_FAILURE, QRY__GEN__NO_QUERY_RESULTS, QRY__GEN__UNPARSEABLE_MODEL_PARAMS, CX_SPECS__INVALID_JSON, CX_SPECS__UNREGISTERED_TYPE, CX_SPECS__MISSING_SPEC, CX_SPECS__MISSING_TYPE, CX_SPECS__UNPARSEABLE_SPEC, ADM__INVALID_CUSTOMER_ID, ADM__INVALID_CORPUS_ID, ADM__INVALID_ENCODER_ID, ADM__INVALID_ROLE_ID, ADM__ROLE_ALREADY_EXISTS, ADM__ONLY_ONE_OWNER_SUPPORTED, ADM__INVALID_PERMISSION, ADM__ROLECREATION_FAILURE, ADM__USER_EMAIL_NOT_AVAIALBLE, ADM__USERNAME_NOT_AVAILABLE, ADM__SIGNUP_MISSING_NAME, ADM__SIGNUP_MISSING_ORG, ADM__SIGNUP_MISSING_EMAIL, ADM__SIGNUP_MISSING_PAYMENT, ADM__SIGNUP_MISSING_PLAN, ADM__SIGNUP_MISSING_PASSWORD, ADM__SIGNUP_INVALID_NAME, ADM__SIGNUP_INVALID_ORG, ADM__SIGNUP_INVALID_EMAIL, ADM__SIGNUP_INVALID_PAYMENT, ADM__SIGNUP_INVALID_PLAN, ADM__SIGNUP_INVALID_PASSWORD, ADM__SIGNUP_INVALID_ACCOUNT_ALIAS, ADM__SIGNUP_INVALID_EMAIL_VALIDATION_CODE, ADM__SIGNUP_MISSING_COUNTRY_CODE, ADM__SIGNUP_ROOT_EMAIL_NOT_AVAILABLE, ADM__CUST_MARK_DELETE_FAILED, ADM__CUST_FAISS_DEALLOC_FAILED, ADM__CUST_ALREADY_ACTIVE, ADM__CUST_REACTIVATE_FAILED, ADM__CUST_ENABLEMENT_FAILED, ADM__CORPUS_LIMIT_REACHED, ADM__STRIPE_CARD_DECLINED, ADM__STRIPE_PROCESSING_ERROR, ADM__EMAIL_VALIDATION_REQUEST_NOT_FOUND, ADM__EMAIL_NOT_VALIDATED, ADM__CHANGE_PLAN__NO_CURRENT_PLAN, ADM__CHANGE_PLAN__REQUIRES_MANUAL_CHANGE, ADM__CHANGE_PLAN__INVALID_PLAN_ID, ADM__CHANGE_PLAN__NO_PAYMENT_SOURCE, ADM__CHANGE_PLAN__INVALID_EFFECTIVE_DATE, ADM__CHANGE_PLAN__CONFLICTING_CHANGE, SCM__MISCONFIGURED_CONNECTION, STATS_DB_READ_FAILURE, VDB__TEXT_READ_FAILURE, REBUILD__LOW_RECALL, REBUILD__INDEX_UPLOAD_FAILURE, REBUILD__UPDATE_JOURNAL_FAILURE, REBUILD__UPDATE_FAISSPARAMS_FAILURE, REBUILD__NO_DATA, REBUILD__EVALUATION, IDX__TRANSIENT_PARTIAL_DELETION_FAILURE, IDX__PERMANENT_PARTIAL_DELETION_FAILURE, CALB__INVALID_JSON, CALB__INVALID_SPEC, CALB__UNREGISTERED_TYPE, CALB__MISSING_SPEC, CALB__MISSING_TYPE, CALB__UNPARSABLE_SPEC]

Default value: OK

statusDetail string

]

document object[]

Array [

id string

metadata object[]

Array [

name string

value string

]

summary object[]

A summary. If using synchronous APIs for querying, the summary will be included directly in this response. However, if using the streaming APIs for query, the summary messages only set the future_id field. Later, as summary results are computed and returned over the stream, the future_id within the summary can be used for correlation.

Array [

text string

The summary text.

lang string

ISO 639 language code of the summary. If the requested language was set to "AUTO", the summary language is the same as the auto-detected language of the query.

chat object

conversationId string

The conversation id of the chat.

turnId string

The id assigned to this query and answer.

status object

code vectaraStatusCode

Default value: OK

statusDetail string

factualConsistency object

score float

The probability that the summary is factually consistent with the results in the ResponseSet.

status object

code vectaraStatusCode

Default value: OK

statusDetail string

done boolean

Determines if the summary is done. false if the summary is in-progress for streaming requests, otherwise true. this only refers to summary text generation, Factual Consistency will come later if requested.

status object[]

Statuses are marked “repeated” for consistency and flexibility. A failed summary should bubble up into the status code of the entire ResponseSet.

Array [

code vectaraStatusCode

Default value: OK

statusDetail string

]

futureId int32

Populated for streaming requests only.

]

futureId int32

Populated for streaming requests only.

]

status object[]

Array [

code vectaraStatusCode

Default value: OK

statusDetail string

]

metrics object

Basic performance metrics that can be attached to a server response.

queryEncodeMs int64

How long it took to encode the query.

retrievalMs int64

How long it took to retrieve relevant results.

userdataRetrievalMs int64

How long it took to retrieve user data.

rerankMs int64

How long it took to rerank the results.

responseSet object

response object[]

Array [

text string

score float

The score used for ranking results. The higher the score, the better the match.

metadata object[]

Array [

name string

value string

]

documentIndex int64

Use this ID to find the document in the ResponseSet.

corpusKey object

customerId int64required

The Customer ID.

corpusId int64required

Possible values: >= 1

The Corpus ID.

semantics CorpusKeySemantics

Possible values: [DEFAULT, QUERY, RESPONSE]

Default value: DEFAULT

Semantics controls the interpretation of the query string by the server, and can be used to override the default semantics assigned in the corpus definition.

DEFAULT: Use corpus-assigned semantics. This is the most common setting.
QUERY: Use query semantics. This is also common.
RESPONSE: Use response semantics. Usage of this is rare.

metadataFilter string

lexicalInterpolationConfig object

Configuration for a linear interpolation.

lambda float

dim object[]

Array [

name stringrequired

weight numberrequired

]

status object[]

Potentially multiple warnings.

Array [

code vectaraStatusCode

Default value: OK

statusDetail string

]

document object[]

Array [

id string

metadata object[]

Array [

name string

value string

]

summary object[]

Array [

text string

The summary text.

lang string

ISO 639 language code of the summary. If the requested language was set to "AUTO", the summary language is the same as the auto-detected language of the query.

chat object

conversationId string

The conversation id of the chat.

turnId string

The id assigned to this query and answer.

status object

code vectaraStatusCode

Default value: OK

statusDetail string

factualConsistency object

score float

The probability that the summary is factually consistent with the results in the ResponseSet.

status object

code vectaraStatusCode

Default value: OK

statusDetail string

done boolean

status object[]

Statuses are marked “repeated” for consistency and flexibility. A failed summary should bubble up into the status code of the entire ResponseSet.

Array [

code vectaraStatusCode

Default value: OK

statusDetail string

]

futureId int32

Populated for streaming requests only.

]

futureId int32

Populated for streaming requests only.

summary object

text string

The summary text.

lang string

ISO 639 language code of the summary. If the requested language was set to "AUTO", the summary language is the same as the auto-detected language of the query.

chat object

conversationId string

The conversation id of the chat.

turnId string

The id assigned to this query and answer.

status object

code vectaraStatusCode

Default value: OK

statusDetail string

factualConsistency object

score float

The probability that the summary is factually consistent with the results in the ResponseSet.

status object

code vectaraStatusCode

Default value: OK

statusDetail string

done boolean

status object[]

Statuses are marked “repeated” for consistency and flexibility. A failed summary should bubble up into the status code of the entire ResponseSet.

Array [

code vectaraStatusCode

Default value: OK

statusDetail string

]

futureId int32

Populated for streaming requests only.

status object[]

A status code at the level of a BatchQueryResponse.

Array [

code vectaraStatusCode

Default value: OK

statusDetail string

]

error object

code int32

message string

details object[]

Array [

@type string

A URL/resource name that uniquely identifies the type of the serialized protocol buffer message. This string must contain at least one "/" character. The last segment of the URL's path must represent the fully qualified name of the type (as in path/google.protobuf.Duration). The name should be in a canonical form (e.g., leading "." is not accepted).

In practice, teams usually precompile into the binary all types that they expect it to use in the context of Any. However, for URLs which use the scheme http, https, or no scheme, one can optionally set up a type server that maps type URLs to message definitions as follows:

If no scheme is provided, https is assumed.
An HTTP GET on the URL must yield a [google.protobuf.Type][]

value in binary format, or produce an error.
Applications are allowed to cache lookup results based on the

URL, or have them precompiled into a binary to avoid any lookup. Therefore, binary compatibility needs to be preserved on changes to types. (Use versioned type names to manage breaking changes.)

Note: this functionality is not currently available in the official protobuf release, and it is not used for type URLs beginning with type.googleapis.com.

Schemes other than http, https (or the empty scheme) might be used with implementation specific semantics.

]

{
  "result": {
    "batchQueryResponse": {
      "responseSet": [
        {
          "response": [
            {
              "text": "string",
              "score": 0,
              "metadata": [
                {
                  "name": "string",
                  "value": "string"
                }
              ],
              "documentIndex": 0,
              "corpusKey": {
                "corpus_id": 12
              }
            }
          ],
          "status": [
            {
              "code": "OK",
              "statusDetail": "string"
            }
          ],
          "document": [
            {
              "id": "string",
              "metadata": [
                {
                  "name": "string",
                  "value": "string"
                }
              ]
            }
          ],
          "summary": [
            {
              "text": "string",
              "lang": "string",
              "chat": {
                "conversationId": "string",
                "turnId": "string",
                "status": {
                  "code": "OK",
                  "statusDetail": "string"
                }
              },
              "factualConsistency": {
                "score": 0,
                "status": {
                  "code": "OK",
                  "statusDetail": "string"
                }
              },
              "done": true,
              "status": [
                {
                  "code": "OK",
                  "statusDetail": "string"
                }
              ],
              "futureId": 0
            }
          ],
          "futureId": 0
        }
      ],
      "status": [
        {
          "code": "OK",
          "statusDetail": "string"
        }
      ],
      "metrics": {
        "queryEncodeMs": 0,
        "retrievalMs": 0,
        "userdataRetrievalMs": 0,
        "rerankMs": 0
      }
    },
    "responseSet": {
      "response": [
        {
          "text": "string",
          "score": 0,
          "metadata": [
            {
              "name": "string",
              "value": "string"
            }
          ],
          "documentIndex": 0,
          "corpusKey": {
            "corpus_id": 12
          }
        }
      ],
      "status": [
        {
          "code": "OK",
          "statusDetail": "string"
        }
      ],
      "document": [
        {
          "id": "string",
          "metadata": [
            {
              "name": "string",
              "value": "string"
            }
          ]
        }
      ],
      "summary": [
        {
          "text": "string",
          "lang": "string",
          "chat": {
            "conversationId": "string",
            "turnId": "string",
            "status": {
              "code": "OK",
              "statusDetail": "string"
            }
          },
          "factualConsistency": {
            "score": 0,
            "status": {
              "code": "OK",
              "statusDetail": "string"
            }
          },
          "done": true,
          "status": [
            {
              "code": "OK",
              "statusDetail": "string"
            }
          ],
          "futureId": 0
        }
      ],
      "futureId": 0
    },
    "summary": {
      "text": "string",
      "lang": "string",
      "chat": {
        "conversationId": "string",
        "turnId": "string",
        "status": {
          "code": "OK",
          "statusDetail": "string"
        }
      },
      "factualConsistency": {
        "score": 0,
        "status": {
          "code": "OK",
          "statusDetail": "string"
        }
      },
      "done": true,
      "status": [
        {
          "code": "OK",
          "statusDetail": "string"
        }
      ],
      "futureId": 0
    },
    "status": [
      {
        "code": "OK",
        "statusDetail": "string"
      }
    ]
  },
  "error": {
    "code": 0,
    "message": "string",
    "details": [
      {
        "@type": "string"
      }
    ]
  }
}

An unexpected error response.

application/json

Schema
Example (from schema)

Schema

code int32

message string

details object[]

Array [

@type string

If no scheme is provided, https is assumed.
An HTTP GET on the URL must yield a [google.protobuf.Type][]

value in binary format, or produce an error.
Applications are allowed to cache lookup results based on the

URL, or have them precompiled into a binary to avoid any lookup. Therefore, binary compatibility needs to be preserved on changes to types. (Use versioned type names to manage breaking changes.)

Note: this functionality is not currently available in the official protobuf release, and it is not used for type URLs beginning with type.googleapis.com.

Schemes other than http, https (or the empty scheme) might be used with implementation specific semantics.

]

{
  "code": 0,
  "message": "string",
  "details": [
    {
      "@type": "string"
    }
  ]
}

StreamQuery

/v1/stream-query

Request​

Header Parameters

Body

Responses​

Request

Responses