not-poma

Chunk size sometimes exceeds max model size

13

``` InvalidRequestError: This model's maximum context length is 4097 tokens, however you requested 4229 tokens (3973 in your prompt; 256 for the completion). Please reduce your prompt; or completion length....

Implement a streaming text splitter

4

Currently, the TextSplitter interface only allows for splitting text into fixed-size chunks and returning the entire list before any queries are run. It would be great if we could split...

Implement a streaming text splitter

2

Currently, the TextSplitter interface only allows for splitting text into fixed-size chunks and returning the entire list before any queries are run. It would be great if we could split...