Queries over your Data

At a high-level, LlamaIndex gives you the ability to query your data for any downstream LLM use case, whether it’s question-answering, summarization, or a component in a chatbot.

This section describes the different ways you can query your data with LlamaIndex, roughly in order of simplest (top-k semantic search), to more advanced capabilities.


A summarization query requires the LLM to iterate through many if not most documents in order to synthesize an answer. For instance, a summarization query could look like one of the following:

  • β€œWhat is a summary of this collection of text?”

  • β€œGive me a summary of person X’s experience with the company.”

In general, a list index would be suited for this use case. A list index by default goes through all the data.

Empirically, setting response_mode="tree_summarize" also leads to better summarization results.

index = GPTListIndex.from_documents(documents)

query_engine = index.as_query_engine(
response = query_engine.query("<summarization_query>")

Queries over Structured Data

LlamaIndex supports queries over structured data, whether that’s a Pandas DataFrame or a SQL Database.

Here are some relevant resources:



Synthesis over Heterogeneous Data

LlamaIndex supports synthesizing across heterogeneous data sources. This can be done by composing a graph over your existing data. Specifically, compose a list index over your subindices. A list index inherently combines information for each node; therefore it can synthesize information across your heterogeneous data sources.

from llama_index import GPTVectorStoreIndex, GPTListIndex
from llama_index.indices.composability import ComposableGraph

index1 = GPTVectorStoreIndex.from_documents(notion_docs)
index2 = GPTVectorStoreIndex.from_documents(slack_docs)

graph = ComposableGraph.from_indices(GPTListIndex, [index1, index2], index_summaries=["summary1", "summary2"])
query_engine = graph.as_query_engine()
response = query_engine.query("<query_str>")


Routing over Heterogeneous Data

LlamaIndex also supports routing over heterogeneous data sources with RouterQueryEngine - for instance, if you want to β€œroute” a query to an underlying Document or a sub-index.

To do this, first build the sub-indices over different data sources. Then construct the corresponding query engines, and give each query engine a description to obtain a QueryEngineTool.

from llama_index import GPTTreeIndex, GPTVectorStoreIndex
from llama_index.tools import QueryEngineTool


# define sub-indices
index1 = GPTVectorStoreIndex.from_documents(notion_docs)
index2 = GPTVectorStoreIndex.from_documents(slack_docs)

# define query engines and tools
tool1 = QueryEngineTool.from_defaults(
    description="Use this query engine to do...",
tool2 = QueryEngineTool.from_defaults(
    description="Use this query engine for something else...",

Then, we define a RouterQueryEngine over them. By default, this uses a LLMSingleSelector as the router, which uses the LLM to choose the best sub-index to router the query to, given the descriptions.

from llama_index.query_engine import RouterQueryEngine

query_engine = RouterQueryEngine.from_defaults(
    query_engine_tools=[tool1, tool2]

response = query_engine.query(
    "In Notion, give me a summary of the product roadmap."


Compare/Contrast Queries

You can explicitly perform compare/contrast queries with a query transformation module within a ComposableGraph.

from llama_index.indices.query.query_transform.base import DecomposeQueryTransform
decompose_transform = DecomposeQueryTransform(
    llm_predictor_chatgpt, verbose=True

This module will help break down a complex query into a simpler one over your existing index structure.


You can also rely on the LLM to infer whether to perform compare/contrast queries (see Multi-Document Queries below).

Multi-Document Queries

Besides the explicit synthesis/routing flows described above, LlamaIndex can support more general multi-document queries as well. It can do this through our SubQuestionQueryEngine class. Given a query, this query engine will generate a β€œquery plan” containing sub-queries against sub-documents before synthesizing the final answer.

To do this, first define an index for each document/data source, and wrap it with a QueryEngineTool (similar to above):

from llama_index.tools import QueryEngineTool, ToolMetadata

query_engine_tools = [
        metadata=ToolMetadata(name='sept_22', description='Provides information about Uber quarterly financials ending September 2022')
        metadata=ToolMetadata(name='june_22', description='Provides information about Uber quarterly financials ending June 2022')
        metadata=ToolMetadata(name='march_22', description='Provides information about Uber quarterly financials ending March 2022')

Then, we define a SubQuestionQueryEngine over these tools:

from llama_index.query_engine import SubQuestionQueryEngine

query_engine = SubQuestionQueryEngine.from_defaults(query_engine_tools=query_engine_tools)

This query engine can execute any number of sub-queries against any subset of query engine tools before synthesizing the final answer. This makes it especially well-suited for compare/contrast queries across documents as well as queries pertaining to a specific document.


Multi-Step Queries

LlamaIndex can also support iterative multi-step queries. Given a complex query, break it down into an initial subquestions, and sequentially generate subquestions based on returned answers until the final answer is returned.

For instance, given a question β€œWho was in the first batch of the accelerator program the author started?”, the module will first decompose the query into a simpler initial question β€œWhat was the accelerator program the author started?”, query the index, and then ask followup questions.


Temporal Queries

LlamaIndex can support queries that require an understanding of time. It can do this in two ways:

  • Decide whether the query requires utilizing temporal relationships between nodes (prev/next relationships) in order to retrieve additional context to answer the question.

  • Sort by recency and filter outdated context.