Optimal Chunk Size for Large Document Summarization

A method to choose the optimal chunk size for large document summarization.

See our blog post for a detailed introduction.

• toy.ipynb shows a toy chunking comparison.

• summarization.ipynb shows a practical large document summarization comparison.

Get connected to see how we could help to improve your LLM pipelines.

By Vectify AI