flink-connector-aws
flink-connector-aws copied to clipboard
[FLINK-35660] Adding parent-child shard ordering for DDB Streams source
Purpose of the change
- Implements parent-child shard ordering in DDB Streams source
- Source enumerator will only assign splits that correspond to a shard when parent shard is either finished, expired or doesn't exist
- Source enumerator also discovers inconsistencies in the shard graph due to describestream and fixes it
Verifying this change
Added unit tests
Significant changes
(Please check any boxes [x] if the answer is "yes". You can first publish the PR and check them afterwards, for convenience.)
- [ ] Dependencies have been added or upgraded
- [ ] Public API has been changed (Public API is any class annotated with
@Public(Evolving)
) - [x] Serializers have been changed
- [x] New feature has been introduced
- If yes, how is this documented? (not applicable / docs / JavaDocs / not documented)