tanuki.py icon indicating copy to clipboard operation
tanuki.py copied to clipboard

Support for streaming responses

Open JackHopkins opened this issue 1 year ago • 0 comments

LLM providers, specifically OpenAI, support streamed responses. We should support this.

Requirements:

  • Iterator typed outputs should be streamable by default.
  • Support 1000s of streamed outputs through last-n context management.
  • TDA should use 'in' syntax with iterator-as-list objects to specify the first N examples that should be streamed

JackHopkins avatar Oct 31 '23 16:10 JackHopkins