semantic-router icon indicating copy to clipboard operation
semantic-router copied to clipboard

Difference of RollingWindowSplitter and StatisticalChunker

Open sopheck opened this issue 6 months ago • 0 comments

Hi, first of all, thank you very much for your work, which is helping me a lot. :heart:

I have a question about the chunkers or splitters that I have not been able to answer. You have moved the chunkers from semantic_routers to semantic_chunkers, but the RollingWindowSplitter still exists in semantic_routers. What is the reason for this? It looks very similar to the StatisticalChunker. What is the difference? And which of the two should I use for texts?

Thank you!

sopheck avatar Jul 31 '24 07:07 sopheck