doris
doris copied to clipboard
[Enhencement](Jdbc catalog) Add two optional properties for jdbc catalog
Proposed changes
Issue Number: close #xxx
Problem summary
- The first property is
only_specified_database
: In the past,Jdbc Catalog
will synchronize all database from source database. Now we add a parameter calledonly_specified_database
to jdbc catalog to allow only the specified database to be synchronized, eg:
create resource if not exists ${resource_name} properties(
"type"="jdbc",
"user"="root",
"password"="123456",
"jdbc_url" = "jdbc:mysql://172.18.0.1:${mysql_port}/doris_test?useSSL=false",
"driver_url" = "https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/jdbc_driver/mysql-connector-java-8.0.25.jar",
"driver_class" = "com.mysql.cj.jdbc.Driver",
"only_specified_database" = "true"
);
if only_specified_database
is true
, jdbc catalog will only synchronize the database which is specified in jdbc_url
.
- The second property is
lower_case_table_names
: This property will synchronize jdbc external data source table names in lower case.
create resource if not exists ${resource_name} properties(
"type"="jdbc",
"user"="doris_test",
"password"="123456",
"jdbc_url" = "jdbc:oracle:thin:@172.18.0.1:${oracle_port}:${SID}",
"driver_url" = "https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/jdbc_driver/ojdbc8.jar",
"driver_class" = "oracle.jdbc.driver.OracleDriver",
"lower_case_table_names" = "true"
);
Checklist(Required)
- [ ] Does it affect the original behavior
- [ ] Has unit tests been added
- [ ] Has document been added or modified
- [ ] Does it need to update dependencies
- [ ] Is this PR support rollback (If NO, please explain WHY)
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
run buildall
TeamCity pipeline, clickbench performance test result: the sum of best hot time: 33.52 seconds stream load tsv: 481 seconds loaded 74807831229 Bytes, about 148 MB/s stream load json: 37 seconds loaded 2358488459 Bytes, about 60 MB/s stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 27 seconds loaded 861443392 Bytes, about 30 MB/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230301153024_clickbench_pr_106608.html
run feut