doris icon indicating copy to clipboard operation
doris copied to clipboard

[Bug] decimal类型超出精度结果不对,streamload导入结果不对

Open chenlinzhong opened this issue 1 year ago • 1 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

Version

create database  if not exists test_db;
CREATE TABLE  test_db.test_tbl (
k1 decimal(1,0) NULL,
v1 decimal(1,0) NULL
) ENGINE=OLAP
DUPLICATE KEY(k1)
DISTRIBUTED BY HASH(k1) BUCKETS 15
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
);
#数据 data.csv
1,1
28,2,
38,3
400,4
5000,5
6000,65
#streamload导入
curl --location-trusted -u ${user}:${password} -H "strict_mode: true" -T data.csv -H "column_separator:,"  http://{host}:{port}/api/test_db/test_tbl/_stream_load

What's Wrong?

2.0+ after load the result is wrong

# 2.1.5-rc02 版本,结果不符合预期,应该是要报错的,目前看插入的时候是按照M的长度从左到右截取的
MySQL [(none)]> select * from test_db.test_tbl;
+------+------+
| k1   | v1   |
+------+------+
|    2 |    2 |
|    4 |    4 |
|    5 |    5 |
|    1 |    1 |
|    3 |    3 |
+------+------+

如果用 insert into 导入是符合预期的,直接报错
MySQL [test_db]> insert into test_tbl values(1,1),(28,2),(38,3),(400,4),(5000,5);
ERROR 1105 (HY000): errCode = 2, detailMessage = (10.81.85.75)[E-255]Arithmetic overflow when converting value 28 from type Int8 to type Decimal(1, 0)

# 1.1.5 版本  符合预期
MySQL [(none)]> select * from test_db.test_tbl;
+------+------+
| k1   | v1   |
+------+------+
|    1 |    1 |
+------+------+

Reason: decimal value is not valid for definition, column=k1, value=28, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=38, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=400, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=5000, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=6000, precision=1, scale=0; . src line [];

#1.2.8版本  符合预期
MySQL [(none)]> select * from test_db.test_tbl;
+------+------+
| k1   | v1   |
+------+------+
|    1 |    1 |
+------+------+
Reason: decimal value is not valid for definition, column=k1, value=28, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=38, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=400, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=5000, precision=1, scale=0; . src line []; 
Reason: decimal value is not valid for definition, column=k1, value=6000, precision=1, scale=0; . src line [];

总结下当前streamload 对decimal超出宽度的行为

  • 有小数: 超出宽度会以最大值或最小值填充
    • 比如 decimal(3,1) 导入123.45结果为99.9 , -123.45结果为-99.9
  • 无小数: 超出宽度则会从左到右截断
    • 比如 deicmal(2,0) 导入123.45 结果为12 , -123.45结果为-12 注:strict_mode 当前不生效

What You Expected?

行为和1.2之前的保持一致

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

chenlinzhong avatar Aug 23 '24 11:08 chenlinzhong

我们正在梳理数字类型cast的逻辑,会对行为进行统一。

jacktengg avatar Aug 27 '24 04:08 jacktengg

We're closing this issue because it hasn't been updated in a while. If you'd like to revive this issue, please reopen it and feel free a maintainer to remove the Stale tag!

github-actions[bot] avatar Sep 12 '25 00:09 github-actions[bot]