Vectara Prompt Engine
The Vectara Prompt Engine empowers Scale users to customize prompt templates that can reference the most relevant text and metadata for use cases that require Retrieval Augmented Generation (RAG). Vectara enables developers to directly add the retrieved documents and their metadata into the prompt generation. Vectara supports Velocity Templates which offer developers a flexible way of customizing prompt templates and enhance the effectiveness of their generative AI applications.
This capability unlocks more advanced workflows and customizations to answer questions about your business data. For example, answer questions based on previous answers, such as with RFI, RFP, and questionnaires. Draft support tickets from user feedback. You can even customize the formatting of results.
Users can override the default prompt text with custom prompt_template
in the
generation
object of a query.
Effective prompts and templates
Effective prompt templates guide LLMs to generate responses that meet specific user needs or objectives in generative AI applications. Define an objective that outlines what you aim to achieve, and provide detailed context to enrich the prompts with nuanced background information to help the AI understand the scenario better. Iteration and refinement are important processes to help you improve the outcome of your prompts.
Reach out to support if you want to modify the default prompt that Vectara uses.
Prompt template design
Prompt template design includes a specific a role
and content
about this role,
which provide context about how you want the role to behave and the kind of
information that you want to retrieve. These values can also specify variables
and functions to provide more nuanced context. You then
add different operations to customize the types of answers you want to retrieve.
Role
The role
specifies the function of the individual or entity that you want to
respond to the prompt. This function indicates the rules, responsibilities, and
perspective of the action being performed. The typical role value is system
.
You then add context to this system with a content
value such as
You are a helpful search assistant.
You could also specify the platform
or environment within which you want the prompt issued and subsequent action
taken and other rules that you want the system to follow.
Content
The content
provides more information about the role and what you expect the
entity to perform. This is crucial when you have loops and iterations in your
prompt templates, like in our example. For each loop, specify the precise action or
feedback desired from the prompt. Capture the context of what needs to be
accomplished or directed. Clear and concise content helps ensure that the
prompt communicates its intent effectively.
Example content can include “You are a helpful search assistant” or “Generate a summary for the query”.
Example prompt template
The following example prompt specifies a role as a helpful search assistant. It then loops through Vectara query results with specific variables and functions. Finally, it generates a summary for the query.
[
{"role": "system", "content": "You are a helpful search assistant."},
#foreach ($qResult in $vectaraQueryResults)
{"role": "user", "content": "Give me the $vectaraIdxWord[$foreach.index] search result."},
{"role": "assistant", "content": "${qResult.getText()}" },
#end
{"role": "user", "content": "Generate a summary for the query '${vectaraQuery}' based on the above results."}
]