dagger icon indicating copy to clipboard operation
dagger copied to clipboard

Published 20 hours ago • verified publisher icon raystack

feat: implement IndexOrderedSplitAssigner

Open Meghajit opened this issue 2 years ago • 1 comments

As part of this issue, we want to add support for configuring a split assigner which can assign splits to source readers in an almost-deterministic order as decided by the user.

The order will be decided by the order of file paths specified as part of the StreamConfig SOURCE_PARQUET_FILE_PATHS.

Meghajit avatar Apr 04 '22 06:04 Meghajit

Removing this from Support for Parquet Files as a Source Milestone as it is a nice to have for the first milestone cc: @prakharmathur82

Meghajit avatar Jun 10 '22 05:06 Meghajit