Azure openAI resources unfortunately differ from standard openAI resources as you can’t generate embeddings unless you use an embedding model. The regions where these models are available can be found here: https://learn.microsoft.com/en-us/azure/cognitive-services/openai/concepts/models#embeddings-models
Furthermore the regions that support embedding models unfortunately don’t support the latest versions (<*>-003) of openAI models, so we are forced to use one region for embeddings and another for the text generation.
import os import json import openai from llama_index.llms import AzureOpenAI from llama_index.embeddings import OpenAIEmbedding from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext import logging import sys logging.basicConfig( stream=sys.stdout, level=logging.INFO ) # logging.DEBUG for more verbose output logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
Here, we setup the embedding model (for retrieval) and llm (for text generation).
Note that you need not only model names (e.g. “text-embedding-ada-002”), but also model deployment names (the one you chose when deploying the model in Azure.
You must pass the deployment name as a parameter when you initialize
api_key = "<insert api key from azure>" api_base = "<insert api base url from azure>" api_type = "azure" api_version = "2023-05-15" llm = AzureOpenAI( model="model name", engine="<insert deployment name from azure>", api_key=api_key, api_base=api_base, api_type=api_type, api_version=api_version, ) # You need to deploy your own embedding model as well as your own chat completion model embed_model = OpenAIEmbedding( model="text-embedding-ada-002", deployment_name="<insert EMBEDDING model deployment name from azure>", api_key=api_key, api_base=api_base, api_type=api_type, api_version=api_version, ) documents = SimpleDirectoryReader("../../data/paul_graham/").load_data()
from llama_index import set_global_service_context service_context = ServiceContext.from_defaults( llm=llm, embed_model=embed_model, ) set_global_service_context(service_context)
index = VectorStoreIndex.from_documents(documents)
> Adding chunk: What I Worked On February 2021 Before col... > Adding chunk: interesting of that type. So I'm not surprised ... > Adding chunk: to be the study of the ultimate truths, compare... > Adding chunk: language called PL/I, and the situation was sim... > Adding chunk: or if there even was a specific moment, but dur... > Adding chunk: an uneasy alliance between two halves, theory a... > Adding chunk: were hundreds of years old. And moreover this ... > Adding chunk: that he'd found such a spectacular way to get o... > Adding chunk: the classes that everyone has to take in fundam... > Adding chunk: students wouldn't require the faculty to teach ... > Adding chunk: or you get merely photographic accuracy, and wh... > Adding chunk: But the Accademia wasn't teaching me anything e... > Adding chunk: In Florence, after paying my part of the rent, ... > Adding chunk: about a new thing called HTML, which was, as he... > Adding chunk: were plenty of earnest students too: kids who "... > Adding chunk: Lisp hacking work was very rare, and I didn't w... > Adding chunk: had done for the popularity of microcomputers. ... > Adding chunk: shopping cart, and I wrote a new site generator... > Adding chunk: seed funding from Idelle's husband Julian. In r... > Adding chunk: for a month," he said, "and it's still not done... > Adding chunk: fun to work on. If all I'd had to do was work o... > Adding chunk: the collar than a picture of the whole shirt. T... > Adding chunk: partly because that's what startups did during ... > Adding chunk: had given us a lot of options when they bought ... > Adding chunk: That's what I should have done, just gone off s... > Adding chunk: buy. Now I could actually choose what neighborh... > Adding chunk: trying to build what it's now clear is about tw... > Adding chunk: dream of building a new Lisp, partly because on... > Adding chunk: me several years to understand the implications... > Adding chunk: seems about as hip. It's not that unprestigiou... > Adding chunk: charge of marketing at a Boston investment bank... > Adding chunk: out "But not me!" and went on with the talk. Bu... > Adding chunk: And neither of them helped founders enough in t... > Adding chunk: fake investors, because they would in a similar... > Adding chunk: batch was so good. You had to be pretty bold to... > Adding chunk: had not originally intended YC to be a full-tim... > Adding chunk: internal software in Arc. But while I continued... > Adding chunk: double from a kidney stone, he suggested that i... > Adding chunk: we agreed to make it a complete changing of the... > Adding chunk: of 2014 painting. I'd never been able to work s... > Adding chunk: his grad student Steve Russell suggested it. Ru... > Adding chunk: defined goal, or it would have been hard to kee... > Adding chunk: pools. It felt like I was doing life right. I r... > Adding chunk: the more exciting.  Italian words for abstr... > Adding chunk: expensive.  Technically the apartment wasn'... > Adding chunk: online means you treat the online version as th... > Adding chunk: logo had been a white V on a red circle, so I m... > Adding chunk: YC was not working with Jessica anymore. We'd b... > [build_index_from_documents] Total LLM token usage: 0 tokens > [build_index_from_documents] Total embedding token usage: 17533 tokens
query = "What is most interesting about this essay?" query_engine = index.as_query_engine() answer = query_engine.query(query) print(answer.get_formatted_sources()) print("query was:", query) print("answer was:", answer)
> [query] Total LLM token usage: 815 tokens > [query] Total embedding token usage: 8 tokens > Source (Doc id: ad03b507-8953-4201-b545-6195c5cfec49): me several years to understand the implications. It meant there would be a whole new generation o... query was: What is most interesting about this essay? answer was: The most interesting thing about this essay is the way the author reflects on the impact of online publishing on their life and career. They discuss how the opening up of the internet to allow for more diverse, and less prestigious, forms of writing allowed them to pursue the kind of writing they were interested in, which was something that had not been possible before. Furthermore, the author acknowledges that their work may not be seen as prestigious, such as Latin, but yet still has a great impact. They further reflect on how their life and career have been shaped by working on these types of projects.