semantic-kernel icon indicating copy to clipboard operation
semantic-kernel copied to clipboard

ITextCompletion Multiple Results (Incl. Streaming)

Open rogerbarreto opened this issue 2 years ago • 0 comments

Motivation and Context

Potential to get multiple results for the same prompt with and without streaming support.

Resolves #526 Closes #526

MultiChatCompletionStream2

Description

This PR introduces abstractions and the capability to get multiple results for the same prompt using the same request including streaming support.

Proposed:

  • Added 2 result abstractions ITextCompletionResult and ITextCompletionStreamingResult
  • Changed current base ITextCompletion interface with the multiple results abstraction capability
  • Added 2 extensions methods to ITextCompletion to keep it retro compatible / avoid major breaking changes
  • Added concrete implementation to: Azure OpenAI / Open AI / Hugging Face completion impl.
  • Added Samples (with and without streaming) how to use multiple results

Breaking potential:

  • Any custom completion implementation will need to implement the new methods in ITextCompletion interface with the ITextCompletionResult/ITextCompletionStreamingResult abstractions. Pretty straightforward and example was added in CustomLLM sample.

Contribution Checklist

  • [x] The code builds clean without any errors or warnings
  • [x] The PR follows SK Contribution Guidelines (https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
  • [x] The code follows the .NET coding conventions (https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions) verified with dotnet format
  • [x] All unit tests pass, and I have added new tests where possible
  • [x] I didn't break anyone :smile:

rogerbarreto avatar May 11 '23 14:05 rogerbarreto