[WIP][FEAT] Azure AI Update

[riedgar-ms]

trafficstars

A very rough draft updating support for Azure AI models. Credit to @hudson-ai for doing the majority of the work creating the new Interpreter classes.

There are complications arising from the fact that Azure AI has two separate APIs, dependent on whether one is working with an OpenAI model, or with one of the other available models. This is likely to be a highly mobile target for a while.

[codecov-commenter]

:warning: Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 21.80974% with 337 lines in your changes missing coverage. Please review.

Project coverage is 59.94%. Comparing base (9fe8b26) to head (1b74c18). Report is 47 commits behind head on main.

Files with missing lines	Patch %	Lines
guidance/models/_openai_base.py	27.04%	178 Missing :warning:
guidance/models/experimental/_vllm.py	20.37%	86 Missing :warning:
guidance/models/_azureai.py	0.00%	68 Missing :warning:
guidance/models/_openai.py	50.00%	5 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1184       +/-   ##
===========================================
+ Coverage   43.44%   59.94%   +16.50%     
===========================================
  Files          72       62       -10     
  Lines        5849     4781     -1068     
===========================================
+ Hits         2541     2866      +325     
+ Misses       3308     1915     -1393     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[riedgar-ms]

@hudson-ai @paulbkoch , I'd appreciate your feedback on this. Dependent on the current 'merge philosophy,' this is almost ready to merge (if we want more, smaller, incremental changes). I believe that this is going to enable gen() calls at least for both OpenAI and 'other' models deployed in Azure. My personal preference would be to merge this, then get the smoke tests working again in CI, and then go back and figure out how to enable constrained decoding where appropriate (along with some smoke test of that functionality).

The major missing piece as compared to the one-pager is a single factory to cover both OpenAI and 'other' models. Now I've spent some time digging into the two APIs, UXs etc., I'm ambivalent as to whether it would be useful at this point in time.

If you'd prefer not to merge until the constrained decoding part can be activated as well, I'm OK with that too.

[hudson-ai]

@riedgar-ms I'm fine merging before figuring out general constrained decoding here

[nchammas]

Could we update the relevant docs for this feature as part of this PR? Or are the existing docs valid as-is?

[hudson-ai]

@nchammas I'm reluctant to add much documentation until this gets some room to breathe -- I'd still consider this a bit experimental (unless you disagree @riedgar-ms!). Could ask an LM to add some docstrings if needed? 😆

[hudson-ai]

@riedgar-ms I merged experimental vLLM support into main, namespaced very very separately (guidance.models.experimental). I just copied and pasted some of the relevant bits of the BaseOpenAIInterpreter over there, basically just dropping the json, regex, and rule implementations, as vLLM has bona-fide guidance support.

Let's leave it as it is and figure out the right way to share code in a follow-up PR? 😄

[riedgar-ms]

I've merged in the changes from main .... hopefully correctly (things were a bit confused in openai.py). I have also fixed the doc strings on the two new factory functions. Getting those done is a fair ask, but I am reluctant to spend too much time on docs yet.