Cannot initialize AzureOpenAI llm

[kblissett]

A minor bug is causing the AzureOpenAI llm __init__ to fail.

Steps to reproduce:

import guidance

guidance.llms.AzureOpenAI("gpt-3.5-turbo")

Error message:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[17], line 3
      1 import guidance
----> 3 guidance.llms.AzureOpenAI("gpt-3.5-turbo")

File [~/medical-writing/.venv/lib/python3.10/site-packages/guidance/llms/_azure_openai.py:34](https://file+.vscode-resource.vscode-cdn.net/Users/kevinblissett/medical-writing/~/medical-writing/.venv/lib/python3.10/site-packages/guidance/llms/_azure_openai.py:34), in AzureOpenAI.__init__(self, model, client_id, authority, caching, max_retries, max_calls_per_min, token, endpoint, scopes, temperature, chat_mode)
     31 if os.path.exists(self._token_cache_path):
     32     self._token_cache.deserialize(open(self._token_cache_path, 'r').read())
---> 34 self._rest_headers["X-ModelType"] = self.model_name

AttributeError: 'AzureOpenAI' object has no attribute '_rest_headers'

[pcliupc]

met the same issue here.

[nasirus]

+1

[mattberns]

+1 Also struggling with how to specify connection to azure. It is prompting me for a client_id...Received same message with and without the token param which is probably used incorrectly. Any idea how to connect this thing?

Getting response when using the program function: AADSTS900144: The request body must contain the following parameter: 'client_id'

guidance.llm = guidance.llms.AzureOpenAI(model='text-davinci-003',
                                    endpoint='https://<my instance>.openai.azure.com',
                                    token=os.environ["<MY KEY FROM AZURE PORTAL>"])

[wwwcojp]

I was able to solve this problem using device code flow. https://learn.microsoft.com/en-us/azure/active-directory/develop/scenario-desktop-acquire-token-device-code-flow?tabs=python https://learn.microsoft.com/en-us/azure/active-directory/develop/scenario-daemon-app-registration

guidance.llm = guidance.llms.AzureOpenAI(model="text-davinci-003",
                endpoint="https://<endpoint url>", client_id="<client id(application id)>", 
                authority="https://login.microsoftonline.com/<Azure AD tenant id>",
                scopes=["https://cognitiveservices.azure.com/.default"])

[somisawa]

@kblissett Normally, Enterprise Application, which contains client_id and client secret, is needed according to guidance/llms/_azure_openai.py If you wanna use Azure OpenAI simply with deployment name like official openai python api, try my solution (#87 )

[williambrach]

any updates regarding support for AZURE_OPEN_AI_API_KEY?

[javier-alvarez]

any updates on fixes for token and end_point?

[slundberg]

Sorry this took a while. This should work now on 0.0.61:

import guidance

llm = guidance.llms.OpenAI(
    'text-davinci-003',
    api_type='azure',
    api_key='sk-...',
    api_base='https://example-endpoint.openai.azure.com',
    api_version='2023-05-15',
    deployment_id='...',
    caching=False
)

program = guidance("""My favorite flavor is{{gen 'flavor' max_tokens=10 stop="."}}""", llm=llm)
program()