Zep Vector Store๏ƒ

A long-term memory store for LLM applications๏ƒ

This notebook demonstrates how to use the Zep Vector Store with LlamaIndex.

About Zep๏ƒ

Zep makes it easy for developers to add relevant documents, chat history memory & rich user data to their LLM appโ€™s prompts.

Note๏ƒ

Zep can automatically embed your documents. The LlamaIndex implementation of the Zep Vector Store utilizes LlamaIndexโ€™s embedders to do so.

Getting Started๏ƒ

Quick Start Guide: https://docs.getzep.com/deployment/quickstart/ GitHub: https://github.com/getzep/zep

# !pip install zep-python

import logging
import sys
from uuid import uuid4

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

import os
import openai

os.environ["OPENAI_API_KEY"] = "sk-..."
openai.api_key = os.environ["OPENAI_API_KEY"]

from llama_index import VectorStoreIndex, SimpleDirectoryReader
from llama_index.vector_stores import ZepVectorStore

INFO:numexpr.utils:Note: NumExpr detected 12 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
Note: NumExpr detected 12 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
INFO:numexpr.utils:NumExpr defaulting to 8 threads.
NumExpr defaulting to 8 threads.

# load documents
documents = SimpleDirectoryReader("../data/paul_graham").load_data()

Create a Zep Vector Store and Index๏ƒ

You can use an existing Zep Collection, or create a new one.

from llama_index.storage.storage_context import StorageContext

zep_api_url = "http://localhost:8000"
collection_name = f"graham{uuid4().hex}"

vector_store = ZepVectorStore(
    api_url=zep_api_url, collection_name=collection_name, embedding_dimensions=1536
)

storage_context = StorageContext.from_defaults(vector_store=vector_store)

index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)

INFO:httpx:HTTP Request: GET http://localhost:8000/healthz "HTTP/1.1 200 OK"
HTTP Request: GET http://localhost:8000/healthz "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: GET http://localhost:8000/api/v1/collection/graham56f41d07312b46e59df8f68cb5fa820c "HTTP/1.1 404 Not Found"
HTTP Request: GET http://localhost:8000/api/v1/collection/graham56f41d07312b46e59df8f68cb5fa820c "HTTP/1.1 404 Not Found"
INFO:llama_index.vector_stores.zep:Collection graham56f41d07312b46e59df8f68cb5fa820c does not exist, will try creating one with dimensions=1536
Collection graham56f41d07312b46e59df8f68cb5fa820c does not exist, will try creating one with dimensions=1536
INFO:httpx:HTTP Request: POST http://localhost:8000/api/v1/collection/graham56f41d07312b46e59df8f68cb5fa820c "HTTP/1.1 200 OK"
HTTP Request: POST http://localhost:8000/api/v1/collection/graham56f41d07312b46e59df8f68cb5fa820c "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: GET http://localhost:8000/api/v1/collection/graham56f41d07312b46e59df8f68cb5fa820c "HTTP/1.1 200 OK"
HTTP Request: GET http://localhost:8000/api/v1/collection/graham56f41d07312b46e59df8f68cb5fa820c "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST http://localhost:8000/api/v1/collection/graham56f41d07312b46e59df8f68cb5fa820c/document "HTTP/1.1 200 OK"
HTTP Request: POST http://localhost:8000/api/v1/collection/graham56f41d07312b46e59df8f68cb5fa820c/document "HTTP/1.1 200 OK"

query_engine = index.as_query_engine()
response = query_engine.query("What did the author do growing up?")

print(str(response))

INFO:httpx:HTTP Request: POST http://localhost:8000/api/v1/collection/graham56f41d07312b46e59df8f68cb5fa820c/search?limit=2 "HTTP/1.1 200 OK"
HTTP Request: POST http://localhost:8000/api/v1/collection/graham56f41d07312b46e59df8f68cb5fa820c/search?limit=2 "HTTP/1.1 200 OK"

Growing up, the author wrote short stories, experimented with programming on an IBM 1401, nagged his father to buy a TRS-80 computer, wrote simple games, a program to predict how high his model rockets would fly, and a word processor. He also studied philosophy in college, but switched to AI after finding it boring. He was involved in building the infrastructure of the web, and eventually started publishing essays online. He also did some painting and learned to cook for groups.

Querying with Metadata filters๏ƒ

from llama_index.schema import TextNode

nodes = [
    TextNode(
        text="The Shawshank Redemption",
        metadata={
            "author": "Stephen King",
            "theme": "Friendship",
        },
    ),
    TextNode(
        text="The Godfather",
        metadata={
            "director": "Francis Ford Coppola",
            "theme": "Mafia",
        },
    ),
    TextNode(
        text="Inception",
        metadata={
            "director": "Christopher Nolan",
        },
    ),
]

collection_name = f"movies{uuid4().hex}"

vector_store = ZepVectorStore(
    api_url=zep_api_url, collection_name=collection_name, embedding_dimensions=1536
)

storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex(nodes, storage_context=storage_context)

INFO:httpx:HTTP Request: GET http://localhost:8000/healthz "HTTP/1.1 200 OK"
HTTP Request: GET http://localhost:8000/healthz "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: GET http://localhost:8000/api/v1/collection/movies398316dc88a045e7a340c46f6eb52625 "HTTP/1.1 404 Not Found"
HTTP Request: GET http://localhost:8000/api/v1/collection/movies398316dc88a045e7a340c46f6eb52625 "HTTP/1.1 404 Not Found"
INFO:llama_index.vector_stores.zep:Collection movies398316dc88a045e7a340c46f6eb52625 does not exist, will try creating one with dimensions=1536
Collection movies398316dc88a045e7a340c46f6eb52625 does not exist, will try creating one with dimensions=1536

INFO:httpx:HTTP Request: POST http://localhost:8000/api/v1/collection/movies398316dc88a045e7a340c46f6eb52625 "HTTP/1.1 200 OK"
HTTP Request: POST http://localhost:8000/api/v1/collection/movies398316dc88a045e7a340c46f6eb52625 "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: GET http://localhost:8000/api/v1/collection/movies398316dc88a045e7a340c46f6eb52625 "HTTP/1.1 200 OK"
HTTP Request: GET http://localhost:8000/api/v1/collection/movies398316dc88a045e7a340c46f6eb52625 "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST http://localhost:8000/api/v1/collection/movies398316dc88a045e7a340c46f6eb52625/document "HTTP/1.1 200 OK"
HTTP Request: POST http://localhost:8000/api/v1/collection/movies398316dc88a045e7a340c46f6eb52625/document "HTTP/1.1 200 OK"

from llama_index.vector_stores.types import ExactMatchFilter, MetadataFilters

filters = MetadataFilters(filters=[ExactMatchFilter(key="theme", value="Mafia")])

retriever = index.as_retriever(filters=filters)
result = retriever.retrieve("What is inception about?")

for r in result:
    print("\n", r.node.text, r.score)

INFO:httpx:HTTP Request: POST http://localhost:8000/api/v1/collection/movies398316dc88a045e7a340c46f6eb52625/search?limit=2 "HTTP/1.1 200 OK"
HTTP Request: POST http://localhost:8000/api/v1/collection/movies398316dc88a045e7a340c46f6eb52625/search?limit=2 "HTTP/1.1 200 OK"

 The Godfather 0.884073254953458