Pinecone Vector Store

Creating a Pinecone Index

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

import pinecone

api_key = "api_key"
pinecone.init(api_key=api_key, environment="us-east1-gcp")

# dimensions are for text-embedding-ada-002
pinecone.create_index("quickstart", dimension=1536, metric="euclidean", pod_type="p1")

pinecone_index = pinecone.Index("quickstart")

Load documents, build the PineconeVectorStore and GPTVectorStoreIndex

from llama_index import GPTVectorStoreIndex, SimpleDirectoryReader
from llama_index.vector_stores import PineconeVectorStore
from IPython.display import Markdown, display

# load documents
documents = SimpleDirectoryReader('../paul_graham_essay/data').load_data()

# initialize without metadata filter
from llama_index.storage.storage_context import StorageContext


vector_store = PineconeVectorStore(
    pinecone_index=pinecone_index
)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = GPTVectorStoreIndex.from_documents(documents, storage_context=storage_context)

# [optional] initialize with metadata filters
# can define filters specific to this vector index (so you can
# reuse pinecone indexes)
vector_store = PineconeVectorStore(
    pinecone_index=pinecone_index,
    metadata_filters={"title": "paul_graham_essay"}
)
storage_context = StorageContext.from_defaults(vector_store=vector_store)

index = GPTVectorStoreIndex.from_documents(documents, storage_context=storage_context)

Query Index

# set Logging to DEBUG for more detailed outputs
query_engine = index.as_query_engine()
response = query_engine.query("What did the author do growing up?")

display(Markdown(f"<b>{response}</b>"))

The author grew up writing short stories and programming on an IBM 1401. He also nagged his father to buy him a TRS-80 microcomputer, which he used to write simple games, a program to predict how high his model rockets would fly, and a word processor. He studied philosophy in college, but found it boring and switched to AI. He reverse-engineered SHRDLU for his undergraduate thesis and wrote a book about Lisp hacking while in grad school. He also worked on teaching himself Lisp, which he used to create a SHRDLU clone for his thesis. During the first year of grad school, he realized that AI, as practiced at the time, was a hoax and that there was an unbridgeable gap between what these programs could do and actually understanding natural language. He looked around to see what he could salvage from the wreckage of his plans, and there was Lisp. He decided to focus on Lisp and wrote a book about Lisp hacking, On Lisp, which was published in 1993. He also wanted to build things, so he pursued a career in Computer Science, an uneasy alliance between two halves, theory and systems.