flink
flink copied to clipboard
[FLINK-33697][state][metrics] Trace RocksDBIncremental remote files download time
This PR builds on top of https://github.com/apache/flink/pull/23908
Brief change log
Please check individual commit messages
Verifying this change
This change has been manually tested and added a unit test that metric has been created.
Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API, i.e., is any changed class annotated with
@Public(Evolving)
: (yes / no) - The serializers: (yes / no / don't know)
- The runtime per-record code paths (performance sensitive): (yes / no / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
- The S3 file system connector: (yes / no / don't know)
Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
CI report:
- d856fd61fa7548d4f8de404ec368e2b7ec4e9cdd Azure: SUCCESS
Bot commands
The @flinkbot bot supports the following commands:-
@flinkbot run azure
re-run the last Azure build
Thanks for the review!
Why only add the download metric? While you are at this, you could have also simply added metrics for, e.g., merge/clip.
That was the scope of the FLIP. Partially because I wanted to keep the scope as small as possible to minimise required efforts. We can always add more metrics in the future.
@flinkbot run azure
I think that's a fair approach. I agree very few users implement state backends and if they do they are knowledgeable enough to easily migrate their code. Moreover we changed this interface rather freely in the past adding and removing arguments to its methods. Therefore +1 for the approach.