langchain
langchain copied to clipboard
Async Support for LLMChainExtractor
Implemented acompress_documents
and changed syntax for compress_documents
slightly to make sync/async functions consistent.
LLMChainExtractor
as implemented in #2915 for use in the ContextualCompressionRetriever
lacked an async method. As compression of retrieved documents is a highly parallelizable task, this was a major performance bottleneck in my tests.
This implementation is consistent to the implementation in the somewhat similar MapReduceDocumentsChain
chain, see https://github.com/hwchase17/langchain/blob/85dae78548ed0c11db06e9154c7eb4236a1ee246/langchain/chains/combine_documents/map_reduce.py#L131 .
In my own tests (standalone as well as in a compression pipeline) inputs/outputs are unchanged and the async-speedup is significant.
fixed the formatting...
Reverted the Sync changes and just added async without the use of apply
as sugested.
New PR here: #3780 .