Completion Prompts Customization

Prompt Setup

Below, we take the default prompts and customize them to always answer, even if the context is not helpful.

from llama_index.prompts import PromptTemplate

text_qa_template_str = (
    "Context information is below.\n"
    "---------------------\n"
    "{context_str}\n"
    "---------------------\n"
    "Using both the context information and also using your own knowledge, "
    "answer the question: {query_str}\n"
    "If the context isn't helpful, you can also answer the question on your own.\n"
)
text_qa_template = PromptTemplate(text_qa_template_str)

refine_template_str = (
    "The original question is as follows: {query_str}\n"
    "We have provided an existing answer: {existing_answer}\n"
    "We have the opportunity to refine the existing answer "
    "(only if needed) with some more context below.\n"
    "------------\n"
    "{context_msg}\n"
    "------------\n"
    "Using both the new context and your own knowledge, update or repeat the existing answer.\n"
)
refine_template = PromptTemplate(refine_template_str)

Using the Prompts

Now, we use the prompts in an index query!

import openai
import os

os.environ["OPENAI_API_KEY"] = "sk-..."
openai.api_key = os.environ["OPENAI_API_KEY"]

from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from llama_index.llms import OpenAI

service_context = ServiceContext.from_defaults(llm=OpenAI(model="text-davinci-003"))

documents = SimpleDirectoryReader("../../data/paul_graham/").load_data()

index = VectorStoreIndex.from_documents(documents, service_context=service_context)

Before Adding Templates

print(index.as_query_engine().query("Who is Joe Biden?"))

 Joe Biden is not mentioned in the context information.

After Adding Templates

print(
    index.as_query_engine(
        text_qa_template=text_qa_template, refine_template=refine_template
    ).query("Who is Joe Biden?")
)

Joe Biden is the 46th President of the United States. He was elected in 2020 and is the first Democratic president since Barack Obama. He previously served as Vice President under Obama from 2009 to 2017.