seatunnel
seatunnel copied to clipboard
[Improve] [SourceConnector] config supports only specific fields
Search before asking
- [X] I had searched in the issues and found no similar issues.
What happened
config should be able to set all type fields for the corresponding component
SeaTunnel Version
2.1.3
SeaTunnel Config
env {
spark.app.name = "SeaTunnel"
spark.executor.instances = 2
spark.executor.cores = 1
spark.executor.memory = "1g"
spark.master = local
}
source {
hive {
pre_sql = "SELECT user_id, fans_count FROM temp.xhs_user_portrait_count limit 10"
result_table_name = "t_table"
}
}
transform {
}
sink {
jdbc {
saveMode = "Overwrite",
truncate="true",
driver = "com.mysql.cj.jdbc.Driver",
url = "jdbc:mysql:///test?rewriteBatchedStatements=true",
isolationLevel="NONE",
user = "",
password = "",
dbTable = "t_test",
customUpdateStmt = "INSERT INTO t_test (user_id,fans_count) values(?,?)"
jdbc.connect_timeout = 10000
jdbc.socket_timeout = 10000
}
}
Running Command
./bin/start-seatunnel-spark.sh --master yarn --deploy-mode client --config ./test.conf
Error Exception
The jdbc sink component does not support truncate, causing the drop table
Flink or Spark Version
truncate is not supported and the setting causes drop table
Java or Scala Version
1.8
Screenshots
No response
Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct