[FLINK-37275][pipeline-connector/mysql] MySQL source connector supports using the base module's source metric statistics

[beryllw]

close FLINK-37275. FLINK-36315 supported the flink-cdc-base module source metric statistics, and FLINK-37231 added the metrics documentation. However, the MySQL source currently does not support source metric statistics.

[beryllw]

@lvyanquan @liuxiao2shf @ruanhang1993 cc.

[beryllw]

CI. https://github.com/beryllw/flink-cdc/actions/runs/13260216008

[beryllw]

CI. https://github.com/beryllw/flink-cdc/actions/runs/13260216008

@leonardBang Please help me retrigger the CI.

[beryllw]

https://github.com/beryllw/flink-cdc/actions/runs/13387956706

[ruanhang1993]

@beryllw Thanks for your contribution. This PR has 2 problems.

1. The metrics in https://github.com/apache/flink-cdc/pull/3619/files relies on the change of states. But mysql cdc connector does not use the framework in flink-cdc-base. So some metrics in SourceReaderMetrics will be wrong.
2. Is it a good idea that MySQL cdc connector introduce the flink-cdc-base dependency in this PR? Maybe we could introduce these metrics when we complete the task to refactor mysql cdc connetor with flink-cdc-base.

[beryllw]

2. Is it a good idea that MySQL cdc connector introduce the flink-cdc-base dependency in this PR? Maybe we could introduce these metrics when we complete the task to refactor mysql cdc connetor with flink-cdc-base.

Agreed, refactoring the mysql cdc connector with flink-cdc-base is the first step.

[yzeng1618]

Hello, may I ask if there are any plans to develop the adaptation of SourceEnumeratorMetrics indicators in the mysql pipeline?

[beryllw]

Hello, may I ask if there are any plans to develop the adaptation of SourceEnumeratorMetrics indicators in the mysql pipeline?

I'm using PR-related code to support MySQL pipeline connector metrics. For the community, we should first focus on refactoring the MySQL CDC connector using flink-cdc-base, which will enable support for the metrics. @lvyanquan cc.

[yzeng1618]

Hello, may I ask if there are any plans to develop the adaptation of SourceEnumeratorMetrics indicators in the mysql pipeline?

I'm using PR-related code to support MySQL pipeline connector metrics. For the community, we should first focus on refactoring the MySQL CDC connector using flink-cdc-base, which will enable support for the metrics. @lvyanquan cc.

Because it is found that the current PR only adapts the metric content of SourceReaderMetrics, and the metrics in SourceEnumeratorMetrics (such as those in the snapshot phase and incremental phase) are still quite important. Do you have any plans for this? If not, I will carry out a simple development on my side.

[beryllw]

Because it is found that the current PR only adapts the metric content of SourceReaderMetrics, and the metrics in SourceEnumeratorMetrics (such as those in the snapshot phase and incremental phase) are still quite important. Do you have any plans for this? If not, I will carry out a simple development on my side.

Is it this file flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/assigners/MySqlSnapshotSplitAssigner.java where the changes should be made? Could you explain in more detail what you want to implement? If I haven't implemented it, please go ahead.

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 60 days if no further activity occurs.