batched-fn icon indicating copy to clipboard operation
batched-fn copied to clipboard

how does this map between request source and response?

Open avnerbarr opened this issue 2 years ago • 3 comments

If I have 2 concurrent requesters which are batched together, how does this know which response to send to which requester?

avnerbarr avatar Dec 22 '21 11:12 avnerbarr

doesn't seem to work correctly

image

avnerbarr avatar Dec 22 '21 11:12 avnerbarr

The handler function must return the results in the same order of the corresponding inputs.

epwalsh avatar Dec 22 '21 18:12 epwalsh

In particular, when a new input is received, evaluate_in_batch() is called, which creates a new channel and then sends the input and the sender to the background batch worker through that new channel. When the batch worker finishes processing a batch, it collects the outputs and the corresponding senders and sends the output back through that new channel. That's where the order of the results matters.

epwalsh avatar Dec 22 '21 18:12 epwalsh