seatunnel
seatunnel copied to clipboard
[Bug][Connector-Kudu] Kudu string columns are auto‑created as CHAR(16) in Doris
Search before asking
- [x] I had searched in the issues and found no similar issues.
What happened
When using SeaTunnel to synchronize data from Kudu to Doris with schema auto‑creation enabled, Kudu STRING columns (no length defined on source) are automatically created in Doris as CHAR(16).
In our production tables, many Kudu columns are STRING and the actual data length is far greater than 16. After SeaTunnel runs, the corresponding Doris columns become CHAR(16), which:
-
Risks data truncation or write failures.
-
Does not match the expected Doris STRING (unbounded) or large VARCHAR type.
SeaTunnel Version
2.3.12
SeaTunnel Config
env {
parallelism = 2
job.mode = "BATCH"
}
source {
kudu {
kudu_masters = "10.xx.xx.xx:7051"
table_name = "kudu_source_table"
plugin_output = "kudu"
enable_kerberos = false
}
}
sink {
Doris {
fenodes = "doris-fe:8030"
username = "root"
password = ""
database = "kudu_sync"
table = "${table_name}" # 目标表按源表名创建
doris.config = {
format = "json"
read_json_by_line = "true"
}
}
}
Kudu source table example
name type
-------------------------
id string
create_date string
user string
office string
...
the auto‑created Doris table
Field Type
------------------------
id char(16)
create_date char(16)
user char(16)
office char(16)
Running Command
${FLINK_HOME}/bin/flink run \
-c org.apache.seatunnel.core.starter.flink.SeaTunnelFlink \
/opt/seatunnel/seatunnel-flink-15-starter.jar \
--config /opt/seatunnel/config/kudu_to_doris.conf \
--name kudu_to_doris
Error Exception
Caused by: org.apache.seatunnel.exception,DorisConnectorException: ErrorCode:Doris-01], ErrorDescription:stream load errorl.
stream load error: [CANCELLEDIIDATA QUALITY ERRORlEncountered unqualifed data, stop processing, see more in http:/xxxxxx/api/_ load_error log?file=_ shard 3/error log insert_stmt 6a406de74872b031-c7e92b80ed7b50bd 6a406de74872b031 c7e92b80ed7b50bd
Zeta or Flink or Spark Version
flink1.16
Java or Scala Version
java8
Screenshots
Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
Code of Conduct
- [x] I agree to follow this project's Code of Conduct