API is not getting my datasource citations

[renatoromao]

Hi,

I already raised two bugs on Azure support but no solutions yet.

We have implemented the same code as you have on the app.py file, tested on Postman and then moved to a C# code.

Since this week, no citations have been displayed which is weird because the app worked early this month without changes.

I'm using this API below. https://[OPEN_AI].openai.azure.com/openai/deployments/gpt-4/extensions/chat/completions?api-version=2023-06-01-preview

The chat playground is working, but when I deploy it as an Azure Web App, there are no documents/answers (same question). And my API neither.

[raffaeler]

I can confirm it works as expected. The proof is the Azure dashboard that, at the end, is using the services.

Are you sending the correct headers? (just grab them from the Azure dashboard) They are fundamentals to make it work.

Also, please be specific on what you are seeing that is not correct or does not work.

[renatoromao]

hi @raffaeler ,

Thanks for your updates!

My main question is why is not working now? :D It was working last week, with the same API, no changes were made since last week. Including new apps deployed through Chat Playground are not working as well, I've tested them today.

As you can see below, I'm using 4 headers (the same headers suggested by Azure).

[raffaeler]

There have been major issue 2-3 weeks ago when they discontinued the old models. Did your issues start after that event?

What is the error returned by the service? Did you try in another region (I am working on East-US)? Did you check the index for your data is still good?

[HuapingLiu]

I got the similar issue this morning, but it is back to normal now.

[renatoromao]

@raffaeler ,

I didn't notice that.

Did your issues start after that event? I recreated new models last week, not old models (gpt-35-turbo-16k, gpt-35-turbo, gpt-4, gpt-4-32k).

What is the error returned by the service? It is returning no citations and a generic answer.

"messages": [
                {
                    "index": 0,
                    "role": "tool",
                    "content": "{\"citations\": [], \"intent\": \"[\\\"[MY QUESTION HERE]\\\"]\"}",
                    "end_turn": false
                },
                {
                    "index": 1,
                    "role": "assistant",
                    "content": "I don't find any information related to '[MY QUESTION]' from the fetched documents. As a result, I am unable to provide an answer.",
                    "end_turn": true
                }
            ]

Did you try in another region (I am working on East-US)? yes, tried now and no luck.

Did you check the index for your data is still good? yes, it is.

[renatoromao]

@HuapingLiu did you make any changes? Which region?

I'm using mainly North Europe.

[HuapingLiu]

East-US. I opened an issue this morning: https://github.com/microsoft/sample-app-aoai-chatGPT/issues/96. I suspected the root issue is the citations are empty, so they did not show up, instead to use some #doc1, #doc2.

I did not do any change, it is back to normal this afternoon.

I suspect the 2023-06-01-preview AZURE_OPENAI_PREVIEW_API_VERSION is not stable.

[raffaeler]

@renatoromao I just repeated the test on East-US. My code (modified Azure AI library and related app) worked even if with a long delay.

@HuapingLiu

I suspect the 2023-06-01-preview AZURE_OPENAI_PREVIEW_API_VERSION is not stable.

I totally agree. There have been several service disruptions in the last weeks.

[sarah-widder]

Hi @raffaeler can you confirm whether you see citations in the playground, but not the web app, for the same questions?

[raffaeler]

Hi @sarah-widder the issues I see are a quite different. It happened as well to me to see answers where the custom content was totally ignored but the answer stated that it could not find any information for my question.

I am sure this is a server-side issue because waiting for hours or a day and repeating the same question succeeded. Please consider that I am using my own code to make questions on the server side which normally works like a charm. All the issues began after you retiring some models. From that day on, the service began to hiccup..

HTH

[gingergenius]

We're having the same problem where there's definitely information related to the question in the retrieved documents, but the model says it could not find anything. We mostly experimented with the web app by running it locally.

[kjerusalmi]

Same problem, did you find a solution ?

[raffaeler]

I don't get your points. If you use the latest preview, there is no need to make any special adjustment. Custom data can be queried very easily by adding the index name/secret in the extensions, and deserializing the answer to get the citations.

3. From my experimentation it appears a const pre-pended role takes precedence before our roleInformation and a {role: system, content: you are ...} system role is completely ignored.

What do you mean here? I always use the system prompt and it does absolutely work fine. I am probably not understanding your tests.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.