Create an LLM

POST 
/v2/llms

Create a new LLM for use with query and chat endpoints

Request

Header Parameters

Request-Timeout integer

Possible values: >= 1

The API will make a best effort to complete the request in the specified seconds or time out.

Request-Timeout-Millis integer

Possible values: >= 1

The API will make a best effort to complete the request in the specified milliseconds or time out.

application/json

Body

oneOf

CreateOpenAILLMRequest

type stringrequired

Default value: openai-compatible

Must be "openai-compatible" for OpenAI and OpenAI-compatible APIs (like Anthropic Claude, Azure OpenAI, etc)

name stringrequired

Name to reference the LLM. This will be used in other endpoints (like query) when using this LLM. If this name conflicts with a global LLM (a LLM that is precofnigured with the Vectara platform), then it will override that LLM for all usages.

description string

Description of the LLM.

model stringrequired

The model name to use with the API (e.g. gpt-4, claude-2, etc). This is used in the API request to the remote LLM provider.

uri urirequired

The URI endpoint for the API (can be OpenAI or any compatible API endpoint)

auth object

Authentication configuration for an LLM

oneOf

BearerAuth

type stringrequired

Default value: bearer

Must be "bearer" for bearer token auth

token stringrequired

The bearer token to use for authentication

HeaderAuth

type stringrequired

Default value: header

Must be "header" for header-based auth

header stringrequired

The header name to use (e.g. x-api-key)

value stringrequired

The header value to use

test_model_parameters object

Any additional parameters that are required for the LLM during the test call.

property name* any

Any additional parameters that are required for the LLM during the test call.

Responses

201

The LLM has been created

application/json

Schema

Schema

id string

Possible values: Value must match regular expression llm_.*

The ID of the LLM.

name string

Name of the LLM.

description string

The description of the LLM.

enabled boolean

Indicates whether the LLM is enabled.

default boolean

If this is the default LLM, it is used in queries when the generator is not specified.

prompts object[]deprecated
List of prompts that the model can use. This is deprecated; see /v2/generation_presets instead.
Array [
id string
Possible values: Value must match regular expression pmt_.*
The ID of the prompt.
name string
Name of the prompt. This is used as the prompt_name in a query.
description string
The description of the prompt.
enabled boolean
Indicates whether the prompt is enabled.
default boolean
Indicates if this prompt is the default prompt used with the LLM.
]

Example (from schema)

{
  "id": "string",
  "name": "string",
  "description": "string",
  "enabled": true,
  "default": true
}

400

Invalid request body

application/json

Schema

Schema

field_errors object
The errors that relate to specific fields in the request.
property name* string
messages message[]
request_id string

The ID of the request that can be used to help Vectara support debug what went wrong.

Example (from schema)

{
  "field_errors": {},
  "messages": [
    "string"
  ],
  "request_id": "string"
}

403

Permissions do not allow creating an LLM

application/json

Schema

Schema

messages message[]

The messages describing why the error occurred.

request_id string

The ID of the request that can be used to help Vectara support debug what went wrong.

Example (from schema)

{
  "messages": [
    "Internal server error."
  ],
  "request_id": "string"
}

Loading...