Azure OpenAI

Azure openAI resources unfortunately differ from standard openAI resources as you can’t generate embeddings unless you use an embedding model. The regions where these models are available can be found here: https://learn.microsoft.com/en-us/azure/cognitive-services/openai/concepts/models#embeddings-models

Furthermore the regions that support embedding models unfortunately don’t support the latest versions (<*>-003) of openAI models, so we are forced to use one region for embeddings and another for the text generation.

import os
import json
import openai
from langchain.llms import AzureOpenAI
from langchain.embeddings import OpenAIEmbeddings
from llama_index import LangchainEmbedding
from llama_index import (
    VectorStoreIndex,
    SimpleDirectoryReader, 
    LLMPredictor,
    ServiceContext
)
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO) # logging.DEBUG for more verbose output
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

This is the API setup for the embedding model

openai.api_type = "azure"
openai.api_base = "<insert api base url from azure>"
openai.api_version = "2022-12-01"
os.environ["OPENAI_API_KEY"] = "<insert api key from azure>"
openai.api_key = os.getenv("OPENAI_API_KEY")

And here you can see the setup for the text generation model, you should note we use deployment name instead of model name as an arguement. This name is the one chosen when deploying the model in Azure.

llm = AzureOpenAI(deployment_name="<insert deployment name from azure>", model_kwargs={
    "api_key": openai.api_key,
    "api_base": openai.api_base,
    "api_type": openai.api_type,
    "api_version": openai.api_version,
})
llm_predictor = LLMPredictor(llm=llm)

# You need to deploy your own embedding model as well as your own chat completion model
embedding_llm = LangchainEmbedding(
    OpenAIEmbeddings(
        model="text-embedding-ada-002",
        deployment="<insert EMBEDDING model deployment name from azure>",
        openai_api_key= openai.api_key,
        openai_api_base=openai.api_base,
        openai_api_type=openai.api_type,
        openai_api_version=openai.api_version,
    ),
    embed_batch_size=1,
)

documents = SimpleDirectoryReader('../paul_graham_essay/data').load_data()

service_context = ServiceContext.from_defaults(
    llm_predictor=llm_predictor,
    embed_model=embedding_llm,
)

index = VectorStoreIndex.from_documents(documents, service_context=service_context)

> Adding chunk: 		

What I Worked On

February 2021

Before col...
> Adding chunk: interesting of that type. So I'm not surprised ...
> Adding chunk: to be the study of the ultimate truths, compare...
> Adding chunk: language called PL/I, and the situation was sim...
> Adding chunk: or if there even was a specific moment, but dur...
> Adding chunk: an uneasy alliance between two halves, theory a...
> Adding chunk: were hundreds of years old.

And moreover this ...
> Adding chunk: that he'd found such a spectacular way to get o...
> Adding chunk: the classes that everyone has to take in fundam...
> Adding chunk: students wouldn't require the faculty to teach ...
> Adding chunk: or you get merely photographic accuracy, and wh...
> Adding chunk: But the Accademia wasn't teaching me anything e...
> Adding chunk: In Florence, after paying my part of the rent, ...
> Adding chunk: about a new thing called HTML, which was, as he...
> Adding chunk: were plenty of earnest students too: kids who "...
> Adding chunk: Lisp hacking work was very rare, and I didn't w...
> Adding chunk: had done for the popularity of microcomputers. ...
> Adding chunk: shopping cart, and I wrote a new site generator...
> Adding chunk: seed funding from Idelle's husband Julian. In r...
> Adding chunk: for a month," he said, "and it's still not done...
> Adding chunk: fun to work on. If all I'd had to do was work o...
> Adding chunk: the collar than a picture of the whole shirt. T...
> Adding chunk: partly because that's what startups did during ...
> Adding chunk: had given us a lot of options when they bought ...
> Adding chunk: That's what I should have done, just gone off s...
> Adding chunk: buy. Now I could actually choose what neighborh...
> Adding chunk: trying to build what it's now clear is about tw...
> Adding chunk: dream of building a new Lisp, partly because on...
> Adding chunk: me several years to understand the implications...
> Adding chunk: seems about as hip.

It's not that unprestigiou...
> Adding chunk: charge of marketing at a Boston investment bank...
> Adding chunk: out "But not me!" and went on with the talk. Bu...
> Adding chunk: And neither of them helped founders enough in t...
> Adding chunk: fake investors, because they would in a similar...
> Adding chunk: batch was so good. You had to be pretty bold to...
> Adding chunk: had not originally intended YC to be a full-tim...
> Adding chunk: internal software in Arc. But while I continued...
> Adding chunk: double from a kidney stone, he suggested that i...
> Adding chunk: we agreed to make it a complete changing of the...
> Adding chunk: of 2014 painting. I'd never been able to work s...
> Adding chunk: his grad student Steve Russell suggested it. Ru...
> Adding chunk: defined goal, or it would have been hard to kee...
> Adding chunk: pools. It felt like I was doing life right. I r...
> Adding chunk: the more exciting.

[2] Italian words for abstr...
> Adding chunk: expensive.

[7] Technically the apartment wasn'...
> Adding chunk: online means you treat the online version as th...
> Adding chunk: logo had been a white V on a red circle, so I m...
> Adding chunk: YC was not working with Jessica anymore. We'd b...
> [build_index_from_documents] Total LLM token usage: 0 tokens
> [build_index_from_documents] Total embedding token usage: 17533 tokens

query = 'What is most interesting about this essay?'
query_engine = index.as_query_engine()
answer = query_engine.query(query)

print(answer.get_formatted_sources())
print('query was:', query)
print('answer was:', answer)

> [query] Total LLM token usage: 815 tokens
> [query] Total embedding token usage: 8 tokens
> Source (Doc id: ad03b507-8953-4201-b545-6195c5cfec49): me several years to understand the implications. It meant there would be a whole new generation o...
query was: What is most interesting about this essay?
answer was: 

The most interesting thing about this essay is the way the author reflects on the impact of online publishing on their life and career. They discuss how the opening up of the internet to allow for more diverse, and less prestigious, forms of writing allowed them to pursue the kind of writing they were interested in, which was something that had not been possible before. Furthermore, the author acknowledges that their work may not be seen as prestigious, such as Latin, but yet still has a great impact. They further reflect on how their life and career have been shaped by working on these types of projects.