Fabric
Fabric copied to clipboard
[Feature request]: Text compression through summarization
What do you need?
Many LLMs users, particularly those who host locally, end up chafing against context size limits. What we could use is a feature to count tokens in a text, split it naturally into parts that fit into the context limit, send those chunks separately to an LLM for it to summarize (including in every chunk the summary of the previous chunks so that it keeps context in mind). That way you could pipe in text of ANY length and pipe it out through any other pattern.