jan icon indicating copy to clipboard operation
jan copied to clipboard

bug: Edge cases with RAG for remote openai compatible models

Open hiro-v opened this issue 1 year ago • 0 comments

Describe the bug In case user turn on Retrieval

  • User might choose their remote OpenAI LLM models, they might need to add embedding API endpoint as well, it will create some edge cases:
    • If user uses Platform OpenAI, they can easily choose between ada-002 or ada-003. They need to add key in order to send file.
    • If user uses Azure OpenAI, they need to create endpoints for embedding and add it to openai models (we have not supported this case). They need to add key in order to send file.
    • If user uses remote OpenAI compatible endpoint, they might only have chat/completion endpoint, no embedding endpoint
      • We might provide user with the choice with local embedding model in this case.

Steps to reproduce Steps to reproduce the behavior:

  1. Enable experimental in Settings
  2. Use OpenAI models
  3. Try not to add key
  4. See error

Expected behavior

  • Clear definition and handler in UI/ Logic in these case

Screenshots If applicable, add screenshots to help explain your issue.

Environment details

  • All, as it's in TS logic

Logs

Additional context Add any other context or information that could be helpful in diagnosing the problem.

hiro-v avatar Feb 08 '24 06:02 hiro-v