Preview capabilities / features

[carlaelisa]

I have a generic question about this solution and how enterprise ready it really is: does this solution uses features or capabilities that are only available in "preview"? I saw some preview api versions but I assume those can be changed to versions that are general available?

Thanks for the feedback.

Best, Carla

[pamelafox]

Are you referring to this preview version for OpenAI? https://github.com/Azure-Samples/azure-search-openai-demo/blob/5e9d142e50a624cd70e42c4c654097e2fb646d36/app/backend/app.py#L298

We are using "2023-07-01-preview" in order to use function_call support. There is actually a more recent version of the API "2023-12-01-preview" which has a slightly different way of specifying function calls, so we may migrate to that soon. Generally, the API versions themselves should remain stable (even when named "-preview"), so it should be fine to use those versions. There was a regression issue a few months ago with one preview version, but that was not by design; the goal is for a version to stay consistent design-wise.

You can see a version listing here: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference

I think there may be a stable version soon, but many customers use -preview versions in production so that they can use new features in them.

[carlaelisa]

Thanks @pamelafox for your answer. Does it mean when using another api_version the backend will still be working? in some organizations using preview version in production is not allowed unfortunately

[carlaelisa]

@pamelafox if you have any insight - i'd appreciate it.

[pamelafox]

I just tested, and the current version of the app requires more recent versions of the SDK, as we use the "tools" functionality of the API. If you use the most recent non-preview version, "2023-05-15", you will get an error message:

{'error': {'message': 'Unrecognized request argument supplied: tools', 'type': 'invalid_request_error', 'param': None, 'code': None}}

You could rewrite the code in chatreadretrieveread.py to avoid using the tools functionality, as it's not strictly necessary. We use it to improve overall answer reliability.

[carlaelisa]

@pamelafox thanks for your message.

As those "tools" functionality are quite new to me, may I ask you where in the chatreadretrieveread.py i can remove it?

[pamelafox]

This is where we specify tools: https://github.com/Azure-Samples/azure-search-openai-demo/blob/d23c7cca71799ef8535389e8033a7a8f2cb09cdb/app/backend/approaches/chatreadretrieveread.py#L128

You would need to remove tools and then change get_search_query to be simpler, as you wouldn't need to parse the full tools response. It would look similar to this, as this is what we had before we used tools: (But lots of other things changed as well, so you can't just copy/paste whole file)

https://raw.githubusercontent.com/Azure-Samples/azure-search-openai-demo/595578c97e24eca8349a317a4940b7d74d20adba/app/backend/approaches/chatreadretrieveread.py

[pamelafox]

I'm going to close this issue since I think I answered your questions. Open a new issue if you have others, thanks!