elasticsearch-net
elasticsearch-net copied to clipboard
Add overload to BulkAll with IAsyncEnumerable
It would be great to see support for IAsyncEnumerable
, because, when pulling data from the database asynchronously, it requires to load data completely into memory in order to use synchronous enumeration (IEnumerable
),
Hi @nesterenko-kv
when pulling data from the database asynchronously, it requires to load data completely into memory in order to use synchronous enumeration (IEnumerable),
The IEnumerable<T>
implementation passed to BulkAll
does not explicitly require loading data completely into memory. For example, it might be an implementation that executes queries against the database to fetch just a page of results at a time.
Were you referring to the loading of each page completely into memory (which currently happens)? There might be some benefit to changing this to support IAsyncEnumerable<T>
, though I think it would also require extending how bulk requests are serialized to the request stream to take full advantage, since bulk operations are currently buffered on the request.