seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Bug] [cup] if task starting,cpu will 100%

Open 13813586515 opened this issue 1 year ago • 0 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

jdbc task,if running,cpu will 100%

SeaTunnel Version

2.3.7

SeaTunnel Config

env {
  parallelism = 1
  job.mode = "BATCH"
}

source {
    Jdbc {
        driver = com.microsoft.sqlserver.jdbc.SQLServerDriver
        url = "jdbc:sqlserver://192.168.011.19:1433;databaseName=BSDATA"
        user = sa
        password = "11111"
        # Define query logic as required
      table_list = [
        {
          table_path = "BSDATA.dbo.SS_COSTINFO"
          # Use query filetr rows & columns
          query = "select * from BSDATA.dbo.SS_COSTINFO where CycleDate_F>='${this_day}'"
        }
        
    ]

    }
}

 

sink {
  Doris {
    fenodes = "192.168.0.168:8030,192.168.0.169:8030,192.168.0.170:8030"
    username = root
    password = "11111"
    database = "ods_bs_data"
    table = "ods_${table_name}"
    # 保证每个文件唯一,拷贝之后文件需要修改此处
    sink.label-prefix = "bs-bsdata-${table_name}-jdbc"
    sink.enable-2pc = "true"
    sink.enable-delete = "true"
    save_mode_create_template="CREATE TABLE IF NOT EXISTS `${database}`.`${table_name}` (${rowtype_primary_key},${rowtype_fields},dt datetime(0) default current_timestamp(0) on update current_timestamp(0)) ENGINE=OLAP UNIQUE KEY (${rowtype_primary_key}) DISTRIBUTED BY HASH (${rowtype_primary_key}) BUCKETS 3 PROPERTIES ( \"replication_allocation\" = \"tag.location.default: 1\",\"in_memory\" = \"false\",\"storage_format\" = \"V2\", \"disable_auto_compaction\" = \"false\")"
    doris.config {
      format = "json"
      read_json_by_line = "true"
    }
  }
}

Running Command

集群

Error Exception

配置了多个任务,一分钟执行一次,当多个任务同时启动,会将cpu打满,但是实际每个任务都是走的增量,数据量都不高,如何控制每个task使用的资源

Zeta or Flink or Spark Version

zeta

Java or Scala Version

1.8

Screenshots

image image

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

13813586515 avatar Aug 29 '24 09:08 13813586515