UPGRADE PROMPT_PREFIX | CITATIONS NON-EXISTING FILES

[phfontes]

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ x ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Ask a question

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?) Windows 10

azd version?

run azd version and copy paste here. 1.0.2

Versions

Mention any other details that might be useful

Hello, I'm having some difficulty defining prompt_prefix functions. I would like to use 100% and only the "power" of gpt to have it answer the queries by referring only to the sources of the indexed documents. Even changing the prompt_prefix, we noticed that it answers questions that have nothing to do with the documents and also in the citation, it cites files that do not exist, I already went to the azure service in cognitive search and in the Indexes tab this document that he created and appears also in the container, this file he created is not there.

I've read some documentation on prompt engineering, but I believe it's in the wording for the prompt_prefix. Could you support me to say what is the correct set of words to avoid these false positives

Thanks! We'll be in touch soon.

[fakoe]

This is probably related to #388

As of today, OpenAI has removed the model for text-davinci-003, which is used for cognitive search queries. You can switch to gpt-35-turbo, but the queries are significantly worse.

I have stumbled across a similar problem today. There are files in my citations, that aren't existing on my blob-storage or in cognitive search. I assume it is due to the changes OpenAI/Microsoft is currenctly doing, since text-davinci-003 has been removed.

[phfontes]

I do not think so. I see that changing the expressions in the prompt_prefix improves the result. But, it still answers things that are not related to the document and cites non-existing files. I believe that is the set of words used.

[fakoe]

I said it is probably related, not that it is the definite cause.

As stated in #389 it seems, the prompt in combination with gpt-35-turbo might not be the best. Which would mean, the reason you have these non-existent file references is caused by a combination of switching to gpt-35-turbo and using the initial prompt as baseline. And that means, your problem is related to OpenAI deprecating davinci-003, as of today.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this issue will be closed.

[pamelafox]

Closing this in favor of the more recent issue with current prompt: https://github.com/Azure-Samples/azure-search-openai-demo/issues/564