flink icon indicating copy to clipboard operation
flink copied to clipboard

[FLINK-33328] Allow TwoPhaseCommittingSink WithPreCommitTopology to alter the type of the Committable - mixin approach

Open pvary opened this issue 2 years ago • 3 comments

What is the purpose of the change

Mixin based implementation for FLIP-372

For now, just checking backward compatibility of the API changes

Brief change log

  • Moving inner classes to main classes:
    • CommitterInitContext from Sink.CommitterInitContext
    • CommittingSinkWriter from TwoPhaseCommittingSink.PrecommittingSinkWriter
    • StatefulSinkWriter from StatefulSink.StatefulSinkWriter
    • WriterInitContext from Sink.InitContext
  • Moving to mixin interfaces:
    • SupportsCommitter from TwoPhaseCommittingSink
    • SupportsWriterState from StatefulSink
    • SupportsPostCommitTopology from WithPostCommitToplogy
    • SupportsPreCommitTopology from WithPreCommitTopology
    • SupportsPreWriteTopology from WithPreWriteTopology
  • Deprecation methods with default implementations
  • Few internal changes to make everything work

Verifying this change

The CI will run all the tests to see if the change breaks anything

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): yes
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? not documented

pvary avatar Dec 12 '23 14:12 pvary

CI report:

  • 98fdb7994b9cee4b560c68ee82ecd45e0059822c Azure: SUCCESS
Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

flinkbot avatar Dec 12 '23 15:12 flinkbot

@flinkbot run azure

pvary avatar Dec 20 '23 16:12 pvary

Thanks @rodmeneses for the review! The PR will merged in several parts, to help the reviewers. The first part is #24022, addressed your comments there.

pvary avatar Jan 09 '24 13:01 pvary

We merged this in several smaller parts

pvary avatar Jan 24 '24 16:01 pvary