PromptHelper

General prompt helper that can help deal with LLM context window token limitations.

At its core, it calculates available context size by starting with the context window size of an LLM and reserve token space for the prompt template, and the output.

It provides utility for β€œrepacking” text chunks (retrieved from index) to maximally make use of the available context window (and thereby reducing the number of LLM calls needed), or truncating them so that they fit in a single LLM call.

class llama_index.indices.prompt_helper.PromptHelper(context_window: int = 3900, num_output: int = 256, chunk_overlap_ratio: float = 0.1, chunk_size_limit: Optional[int] = None, tokenizer: Optional[Callable[[str], List]] = None, separator: str = ' ', max_input_size: Optional[int] = None, embedding_limit: Optional[int] = None, max_chunk_overlap: Optional[int] = None)

Prompt helper.

General prompt helper that can help deal with LLM context window token limitations.

At its core, it calculates available context size by starting with the context window size of an LLM and reserve token space for the prompt template, and the output.

It provides utility for β€œrepacking” text chunks (retrieved from index) to maximally make use of the available context window (and thereby reducing the number of LLM calls needed), or truncating them so that they fit in a single LLM call.

Parameters

  • context_window (int) – Context window for the LLM.

  • num_output (int) – Number of outputs for the LLM.

  • chunk_overlap_ratio (float) – Chunk overlap as a ratio of chunk size

  • chunk_size_limit (Optional[int]) – Maximum chunk size to use.

  • tokenizer (Optional[Callable[[str], List]]) – Tokenizer to use.

  • separator (str) – Separator for text splitter

  • max_input_size (int) – deprecated, now renamed to context_window

  • embedding_limit (int) – deprecated, now consolidated with chunk_size_limit

  • max_chunk_overlap (int) – deprecated, now configured via chunk_overlap_ratio

classmethod from_llm_metadata(llm_metadata: LLMMetadata, chunk_overlap_ratio: float = 0.1, chunk_size_limit: Optional[int] = None, tokenizer: Optional[Callable[[str], List]] = None, separator: str = ' ', max_input_size: Optional[int] = None, embedding_limit: Optional[int] = None, max_chunk_overlap: Optional[int] = None) → PromptHelper

Create from llm predictor.

This will autofill values like context_window and num_output.

get_text_splitter_given_prompt(prompt: Prompt, num_chunks: int = 1, padding: int = 5) → TokenTextSplitter

Get text splitter configured to maximally pack available context window, taking into account of given prompt, and desired number of chunks.

repack(prompt: Prompt, text_chunks: Sequence[str], padding: int = 5) → List[str]

Repack text chunks to fit available context window.

This will combine text chunks into consolidated chunks that more fully β€œpack” the prompt template given the max_input_size.

truncate(prompt: Prompt, text_chunks: Sequence[str], padding: int = 5) → List[str]

Truncate text chunks to fit available context window.