seatunnel
seatunnel copied to clipboard
[Bug] [cup] if task starting,cpu will 100%
Search before asking
- [X] I had searched in the issues and found no similar issues.
What happened
jdbc task,if running,cpu will 100%
SeaTunnel Version
2.3.7
SeaTunnel Config
env {
parallelism = 1
job.mode = "BATCH"
}
source {
Jdbc {
driver = com.microsoft.sqlserver.jdbc.SQLServerDriver
url = "jdbc:sqlserver://192.168.011.19:1433;databaseName=BSDATA"
user = sa
password = "11111"
# Define query logic as required
table_list = [
{
table_path = "BSDATA.dbo.SS_COSTINFO"
# Use query filetr rows & columns
query = "select * from BSDATA.dbo.SS_COSTINFO where CycleDate_F>='${this_day}'"
}
]
}
}
sink {
Doris {
fenodes = "192.168.0.168:8030,192.168.0.169:8030,192.168.0.170:8030"
username = root
password = "11111"
database = "ods_bs_data"
table = "ods_${table_name}"
# 保证每个文件唯一,拷贝之后文件需要修改此处
sink.label-prefix = "bs-bsdata-${table_name}-jdbc"
sink.enable-2pc = "true"
sink.enable-delete = "true"
save_mode_create_template="CREATE TABLE IF NOT EXISTS `${database}`.`${table_name}` (${rowtype_primary_key},${rowtype_fields},dt datetime(0) default current_timestamp(0) on update current_timestamp(0)) ENGINE=OLAP UNIQUE KEY (${rowtype_primary_key}) DISTRIBUTED BY HASH (${rowtype_primary_key}) BUCKETS 3 PROPERTIES ( \"replication_allocation\" = \"tag.location.default: 1\",\"in_memory\" = \"false\",\"storage_format\" = \"V2\", \"disable_auto_compaction\" = \"false\")"
doris.config {
format = "json"
read_json_by_line = "true"
}
}
}
Running Command
集群
Error Exception
配置了多个任务,一分钟执行一次,当多个任务同时启动,会将cpu打满,但是实际每个任务都是走的增量,数据量都不高,如何控制每个task使用的资源
Zeta or Flink or Spark Version
zeta
Java or Scala Version
1.8
Screenshots
Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct