flink-cdc icon indicating copy to clipboard operation
flink-cdc copied to clipboard

[FLINK-37855-][cdc-connector-mysql] Reduce the time cost in MySqlSchemaUtils#listTables

Open zhyayun opened this issue 9 months ago • 2 comments

https://issues.apache.org/jira/browse/FLINK-37855 MySqlSchemaUtils#listTables and TableDiscoveryUtils#listTables is called to get the table names of all tables that need to be processed, However, in scenarios involving many databases and tables, this method call will require querying the database many times, resulting in a long processing time. The optimization idea is to use mysql's metadata information, a connection to get all the library table information.

zhyayun avatar May 30 '25 08:05 zhyayun

I think the key point is reusing the connection, instead of changing the query sql. So passing Connection to listTables() method will be better.

lvyanquan avatar Jun 17 '25 07:06 lvyanquan

This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 60 days if no further activity occurs.

github-actions[bot] avatar Oct 16 '25 00:10 github-actions[bot]