seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[jdbc][jdbc parameter] Muilt Table Sync Use same jdbc connection properties

Open itcrazy0918 opened this issue 2 years ago • 1 comments

Search before asking

  • [X] I had searched in the feature and found no similar feature requirement.

Description

The database connection information in the JDBC source table & sink table needs to be written repeatedly, and cannot be configured only once

suggest Solution: extract it into env module for unified definition or external parameter reference

Usage Scenario

When synchronizing multiple tables, avoid repeatedly writing database connection information, and configure it only once eg :

env { spark.app.name = "sync_base_tables" spark.executor.instances = 1 spark.executor.cores = 1 spark.executor.memory = "1g" }

jdbcconfig { driver = "com.mysql.jdbc.Driver" url = "jdbc:mysql://127.0.0.1:3306/testdb" user = "root" password = "123456" }

xxxconfig{ ...... }

source { jdbc {  ${jdbcconfig} table = "location" result_table_name = "location" jdbc.fetchsize = 500 jdbc.batchsize = 500 } jdbc { ${jdbcconfig} table = "subsystem" result_table_name = "subsystem" jdbc.fetchsize = 500 jdbc.batchsize = 500 } jdbc { ${jdbcconfig} table = "me_alarm_type" result_table_name = "tablec" jdbc.fetchsize = 500 jdbc.batchsize = 500 } } .......

Related issues

there is not another issue associated with this?

Are you willing to submit a PR?

  • [X] Yes I am willing to submit a PR!

Code of Conduct

itcrazy0918 avatar Jul 22 '22 08:07 itcrazy0918

This function is very similar to Hocon's Substitutions. You can refer to SeaTunnel's existing Config library to see if it can be implemented directly.

Hisoka-X avatar Jul 27 '22 02:07 Hisoka-X

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] avatar Apr 05 '23 00:04 github-actions[bot]