azure-search-openai-demo icon indicating copy to clipboard operation
azure-search-openai-demo copied to clipboard

feature: automatic number of document as expert setting

Open cforce opened this issue 1 year ago • 1 comments

Instead of setting a fixed number of documents to be injected into the prompt, dynamically calculate this based on the user's configuration of "Max Length of a System Response" in the expert settings. Allow users to set the document count to "auto" and prompt them to configure the "Max Length of a System Response," with a default value provided.

The number of documents that can be injected into the prompt should be based on the formula:

#Max Response Tokens = #Prompt Tokens + #User Message Tokens + #Document Injected Tokens + #Response Message Tokens

Given:

  • #Response Message Tokens
  • #Max Response Tokens
  • #Prompt Tokens

variables:

  • #User Message Tokens

The process should iterate over the ranked and ordered document list, adding complete documents (or pages) one by one to the prompt until the condition #Max Response Tokens <= 0 is met.

cforce avatar Oct 22 '24 09:10 cforce