Playground

# My OpenAI Key
import os
import openai

os.environ["OPENAI_API_KEY"] = "sk-...."
openai.api_key = os.environ["OPENAI_API_KEY"]

# Hide logs
import logging

logger = logging.getLogger()
logger.setLevel(logging.CRITICAL)

Setup

Generate some example Documents

from llama_index import download_loader
from llama_index.indices.vector_store import VectorStoreIndex
from llama_index.indices.tree.base import TreeIndex

WikipediaReader = download_loader("WikipediaReader")

loader = WikipediaReader()
documents = loader.load_data(pages=["Berlin"])

Create a list of any sort of indices (custom LLMs, custom embeddings, etc)

indices = [
    VectorStoreIndex.from_documents(documents),
    TreeIndex.from_documents(documents),
]

Using the Playground

Initialize with indices

from llama_index.playground import Playground

playground = Playground(indices=indices)

result_df = playground.compare("What is the population of Berlin?")

Query:
What is the population of Berlin?

VectorStoreIndex, retriever mode = default

The population of Berlin is approximately 3.7 million inhabitants.

TreeIndex, retriever mode = select_leaf

It is not possible to answer this question with the given context information.

TreeIndex, retriever mode = select_leaf_embedding

The population of Berlin is approximately 3.7 million inhabitants.

TreeIndex, retriever mode = all_leaf


The population of Berlin is approximately 3.75 million inhabitants. This population has been shaped by the city's turbulent history, with Jewish emigration during the 1930s, the destruction of the city during World War II, and the division of the city into East and West Berlin during the Cold War. Since the reunification of Germany in 1990, Berlin has seen a surge in population growth, with many people from other parts of Germany and the world moving to the city. At the end of 2019, the population of Berlin was estimated to be around 3.75 million inhabitants. The city is home to a diverse religious population, with the faithful of the different religions and denominations maintaining many places of worship in Berlin, including eight parishes of the Independent Evangelical Lutheran Church, 36 Baptist congregations, 29 New Apostolic Churches, 15 United Methodist churches, eight Free Evangelical Congregations, four Churches of Christ, Scientist (1st, 2nd, 3rd, and 11th), six congregations of the Church of Jesus Christ of Latter-day Saints, an Old Catholic church, an Anglican church, more than 80 mosques, ten synagogues, and two Buddhist temples. Berlin is also home to a large number of immigrants from around the world, with 48 percent of the residents under the age of 15 having a migration background in 2017. Berlin is a major economic center in Europe, with many international companies and organizations based in the city, such as the Fraunhofer Society, the Leibniz Association, the Helmholtz Association, and the Max Planck Society, as well as a large number of tourists visiting each year. The city is well-connected to the rest of Germany and Europe through its extensive road, rail, and air transport networks, making it an attractive destination for business and leisure travelers alike. It is also home to a number of renowned research institutions, universities, and medical schools, as well as seven symphony orchestras, including the world-renowned Berlin Philharmonic Orchestra, the Konzerthausorchester Berlin, and the Haus der Kulturen der Welt. Berlin is home to a vibrant cultural and entertainment scene, with a diverse range of cuisine, including Michelin-starred restaurants, vegetarian and vegan offerings, street food, and international cuisine, as well as a variety of botanical gardens, zoos, and other recreational activities. This makes it an attractive destination for people from all over the world. Berlin is also home to two zoos, the Botanischer Garten, the Tiergarten park, and the GΓ€rten der Welt, as well as many cafΓ©s, street musicians, beach bars, flea markets, and boutique shops. Berlin has established a high-profile as a host city of major international sporting events, such as the 1936 Summer Olympics, the 2006 FIFA World Cup final, the IAAF World Championships in Athletics, the Basketball Euroleague Final Four, the UEFA Champions League Final, and the 2023 Special Olympics World Summer Games. It is also home to several professional sports teams, such as Hertha BSC, and has a large Olympic training center.

TreeIndex, retriever mode = root

The population of Berlin is 3.7 million within city limits and 4.5 million in its urban area.


Ran 5 combinations in total.

result_df
Index Retriever Mode Output Duration Prompt Tokens Completion Tokens Embed Tokens
0 VectorStoreIndex default \nThe population of Berlin is approximately 3.... 2.525580 1786 13 7
1 TreeIndex select_leaf \nIt is not possible to answer this question w... 5.536037 4732 115 0
2 TreeIndex select_leaf_embedding \nThe population of Berlin is approximately 3.... 5.426232 897 13 9146
3 TreeIndex all_leaf \n\nThe population of Berlin is approximately ... 238.278128 27291 5035 0
4 TreeIndex root \nThe population of Berlin is 3.7 million with... 3.375349 558 23 0

Initialize with Documents

Automatically construct the playground using a vector, tree, and summary index

# Uses documents in a preset list of indices
playground = Playground.from_docs(documents=documents)