microsoft
Possible improvements for FunctionAggregatePartitionedResultsAsync
My team has been working on our own infrastructure for sending large prompts to GPT.
These prompts are typically in the form of large markdown tables. For instance, we might have a large markdown table and our prompt might also include: "make every letter uppercase." We would break these rows into smaller chunks or units, typically a subset of rows. As each chunk is completed by GPT, we render the results to the UI.
It seems we could do the chunking on our end and pass the chunks to FunctionAggregatePartitionedResultsAsync()
I have a question:
Could a new method be introduced, similar to FunctionAggregatePartitionedResultsAsync()? Currently, this method iterates through all the partitionInput, and we are interested in context.Variables.ToString(). Could that somehow be yield returned to the caller and eventually be added to the context? Currently, we have to wait for all the chunks to be completed before rendering anything to the UI.
We are looking at this as a part of multi-modal support, for now we suggest splitting the large input and call the function multiple times.
Check out 1.0 release to see if this solves your ask - https://devblogs.microsoft.com/semantic-kernel/semantic-kernel-v1-0-1-has-arrived-to-help-you-build-agents/
