flink-cdc icon indicating copy to clipboard operation
flink-cdc copied to clipboard

[cdc-connector][mysql]Mysql add numRecordsOutBySnapshot and numRecordsOutByIncremental metrics

Open ChengJie1053 opened this issue 1 year ago • 3 comments

numRecordsOutBySnapshot metrics collects snapshot statistics

numRecordsOutByIncremental metrics collects incremental statistics

numRecordsOutByIncremental has three metrics, respectively is numRecordsOutByIncrementalInsert, numRecordsOutByIncrementalUpdate, numRecordsOutByIncrementalDelete

image

2024.7.12 change metrics name,change numRecordsOutByIncrementalInsert to numRecordsOutByDataChangeEventInsert image

ChengJie1053 avatar Jul 06 '24 10:07 ChengJie1053

Seems we can't come up with a "best" metric design for everyone: Some may want a task-level unified record counter, while some may want fine-grained counters for each table. Or one may want to disable it entirely to avoid any extra cost.

I wonder if we should add an option to control metric behavior? cc @ruanhang1993

yuxiqian avatar Jul 16 '24 11:07 yuxiqian

@ChengJie1053 cc. @yuxiqian Is thera any changes or progress? If not, I want to create new PR with my suggestion at next week. (I need this feature.)

SML0127 avatar Aug 10 '24 07:08 SML0127

This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs.

github-actions[bot] avatar Oct 10 '24 00:10 github-actions[bot]

This pull request has been closed because it has not had recent activity. You could reopen it if you try to continue your work, and anyone who are interested in it are encouraged to continue work on this pull request.

github-actions[bot] avatar Nov 09 '24 00:11 github-actions[bot]