seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Bug][Connector-Kudu] Kudu string columns are auto‑created as CHAR(16) in Doris

Open yzeng1618 opened this issue 3 weeks ago • 0 comments

Search before asking

  • [x] I had searched in the issues and found no similar issues.

What happened

When using SeaTunnel to synchronize data from Kudu to Doris with schema auto‑creation enabled, Kudu STRING columns (no length defined on source) are automatically created in Doris as CHAR(16).

In our production tables, many Kudu columns are STRING and the actual data length is far greater than 16. After SeaTunnel runs, the corresponding Doris columns become CHAR(16), which:

  • Risks data truncation or write failures.

  • Does not match the expected Doris STRING (unbounded) or large VARCHAR type.

SeaTunnel Version

2.3.12

SeaTunnel Config

env {
  parallelism = 2
  job.mode = "BATCH"
}

source {
  kudu {
    kudu_masters = "10.xx.xx.xx:7051"
    table_name = "kudu_source_table"
    plugin_output = "kudu"
    enable_kerberos = false
  }
}

sink {
  Doris {
    fenodes = "doris-fe:8030"
    username = "root"
    password = ""
    database = "kudu_sync"
    table = "${table_name}"           # 目标表按源表名创建
    doris.config = {
      format = "json"
      read_json_by_line = "true"
    }
  }
}


Kudu source table example
name                type
-------------------------
id               string
create_date   string
user     string
office      string
...

the auto‑created Doris table
Field              Type
------------------------
id              char(16)
create_date  char(16)
user    char(16)
office     char(16)

Running Command

${FLINK_HOME}/bin/flink run \
  -c org.apache.seatunnel.core.starter.flink.SeaTunnelFlink \
  /opt/seatunnel/seatunnel-flink-15-starter.jar \
  --config /opt/seatunnel/config/kudu_to_doris.conf \
  --name kudu_to_doris

Error Exception

Caused by: org.apache.seatunnel.exception,DorisConnectorException: ErrorCode:Doris-01], ErrorDescription:stream load errorl.
stream load error: [CANCELLEDIIDATA QUALITY ERRORlEncountered unqualifed data, stop processing, see more in http:/xxxxxx/api/_ load_error log?file=_ shard 3/error log insert_stmt 6a406de74872b031-c7e92b80ed7b50bd 6a406de74872b031 c7e92b80ed7b50bd

Zeta or Flink or Spark Version

flink1.16

Java or Scala Version

java8

Screenshots

Image

Are you willing to submit PR?

  • [x] Yes I am willing to submit a PR!

Code of Conduct

yzeng1618 avatar Dec 10 '25 06:12 yzeng1618