langchain icon indicating copy to clipboard operation
langchain copied to clipboard

Async Support for LLMChainExtractor

Open jphme opened this issue 1 year ago • 1 comments

Implemented acompress_documents and changed syntax for compress_documents slightly to make sync/async functions consistent.

LLMChainExtractor as implemented in #2915 for use in the ContextualCompressionRetriever lacked an async method. As compression of retrieved documents is a highly parallelizable task, this was a major performance bottleneck in my tests.

This implementation is consistent to the implementation in the somewhat similar MapReduceDocumentsChain chain, see https://github.com/hwchase17/langchain/blob/85dae78548ed0c11db06e9154c7eb4236a1ee246/langchain/chains/combine_documents/map_reduce.py#L131 .

In my own tests (standalone as well as in a compression pipeline) inputs/outputs are unchanged and the async-speedup is significant.

jphme avatar Apr 26 '23 12:04 jphme

fixed the formatting...

jphme avatar Apr 26 '23 19:04 jphme

Reverted the Sync changes and just added async without the use of apply as sugested. New PR here: #3780 .

jphme avatar Apr 29 '23 10:04 jphme