paper-qa
paper-qa copied to clipboard
Summarisation should use batch methods
Currently summarisation makes one call to the underlying LLM per document to summarise.
Instead aget_evidence should use batch methods (agenerate/aapply) to produce a single LLM call.
This could provide significant speedups for local LLMs.