flink icon indicating copy to clipboard operation
flink copied to clipboard

[FLINK-36001][state] Store operator name and UID in state metadata

Open gaborgsomogyi opened this issue 5 months ago • 3 comments

What is the purpose of the change

Normally users can define an operator in a Flink application like this where UID and name can be assigned to it:

stream.map(new ValueProcessUDF())
       .uid(“value-process-uid”)
       .name(“value-process-name”)

At the moment checkpoint metadata file is not containing operator UID and name which makes hard for users to find out what is the human readable intention for a specific operator. Please see FLIP-474 further details.

In this PR I've added the 2 mentioned fields into metadata file.

Brief change log

Added operator UID and name to metadata file.

Verifying this change

  • Existing + modified + new automated tests
  • Manually on cluster (read, write, operator UID change)

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: yes
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: yes
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

gaborgsomogyi avatar Aug 29 '24 09:08 gaborgsomogyi