numaflow icon indicating copy to clipboard operation
numaflow copied to clipboard

feat: Stream mapper implementation of `MapStreamBatchFn`. Fixes #1688

Open magelisk opened this issue 10 months ago • 0 comments

This implements backend for updated stream mapper (MapStreamBatchFn) implemented here: https://github.com/numaproj/numaflow-go/pull/129

It allows stream mapper to have readBatchSize > 1.

  • If readBatchSize == 1, the capability today is executed as-is
  • if readBatchSize >= 1, the newer interface will be used.

Individual langauge servicers can be implemetned to have the default handler for MapStreamBatchFn simply iterate over the one-item implementation so that it should be completely backwards compatible for everyone. But now those who wish to implement handling more than datum item at a time, the capability is now exposed

Fixes: #1688 and partial fix of #1564

magelisk avatar Apr 27 '24 14:04 magelisk