LLamaSharp icon indicating copy to clipboard operation
LLamaSharp copied to clipboard

[BUG]: ChatSession unnecessarily prevents arbitrary conversation interleaving

Open lostmsu opened this issue 1 year ago • 1 comments

Description

With asynchronous flows user might ask 2 questions in succession before model has time to answer the first one. This ends up in the following chat pattern:

user: multiply 2 by 3 user: then divide by -1 assistant: 6 assistant: -6

Unfortunately, for some reason ChatSession prevents this while LLAMA 3 instruct models handle such scenario without any issues.

Reproduction Steps

var history = new ChatHistory();
history.AddMessage(AuthorRole.User, "multiply 2 by 3");
history.AddMessage(AuthorRole.User, "then divide by -1");
history.AddMessage(AuthorRole.Assistant, "6");
history.AddMessage(AuthorRole.Assistant, "-6");

Throws ArgumentException twice

Environment & Configuration

  • Operating system: Win 11
  • .NET runtime version: 8
  • LLamaSharp version: 0.14
  • CUDA version (if you are using cuda backend): 12
  • CPU & GPU device: N/A

Known Workarounds

No response

lostmsu avatar Jul 19 '24 00:07 lostmsu

This issue has been automatically marked as stale due to inactivity. If no further activity occurs, it will be closed in 7 days.

github-actions[bot] avatar Apr 30 '25 00:04 github-actions[bot]