matrixone
matrixone copied to clipboard
matrixorigin
[Bug]: create index on table report 'internal error: Invalid alloc size 1147486208'.
Is there an existing issue for the same bug?
- [X] I have checked the existing issues.
Environment
- Version or commit-id (e.g. v0.1.0 or 8b23a93): b49cc70872b55f58b368c7c4c91b33c2a6f5057b
- Hardware parameters:
- OS type:
- Others:
Actual Behavior
Expected Behavior
create index success!
Steps to Reproduce
data scale: tpch 100G
create index indexTest on partsupp(ps_availqty);
Additional information
No response
这个问题和远宁沟通了，这个要彻底解决要等龙冉把 spills 做好。然后把算子都修改为 1、 一个批次不能超过8192 2、 有些地方，比如join，需要保持数据的。如果超过数据量要持久化， @Ariznawlll @daviszhen @aunjgr
[20260226] 大数据测试出现了这个问题；
job url（queries 1y -> write 1y test）：
https://github.com/matrixorigin/mo-nightly-regression/actions/runs/8029862130/job/21947111382
log:http://175.178.192.213:30088/explore?panes=%7B%22AAL%22:%7B%22datasource%22:%22loki%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%7Bnamespace%3D%5C%22branch-big-data-nightly-ead69b4%5C%22%7D%22,%22queryType%22:%22range%22,%22datasource%22:%7B%22type%22:%22loki%22,%22uid%22:%22loki%22%7D,%22editorMode%22:%22builder%22%7D%5D,%22range%22:%7B%22from%22:%22now-2d%22,%22to%22:%22now%22%7D%7D%7D&schemaVersion=1&orgId=1
https://github.com/matrixorigin/matrixone/issues/12956 https://github.com/matrixorigin/matrixone/issues/12021 https://github.com/matrixorigin/MO-Cloud/issues/1921
都是相似issue，未处理。
等load hung住的问题fix了之后才能测
- https://github.com/matrixorigin/matrixone/pull/15609
commit：28682fdf988e1906bef3b869b13702c6790ea6bd 1亿数据量本地测试： test case：
create table if not exists big_data_test.table_basic_for_load_100M( col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 tinyint unsigned, col6 smallint unsigned, col7 int unsigned, col8 bigint unsigned, col9 float, col10 double, col11 varchar(255), col12 Date, col13 DateTime, col14 timestamp, col15 bool, col16 decimal(16,6), col17 text, col18 json, col19 blob, col20 binary(255), col21 varbinary(255), col22 vecf32(3), col23 vecf32(3), col24 vecf64(3), col25 vecf64(3));
load data url s3option {'endpoint'='******,'access_key_id'='******','secret_access_key'='******','bucket'='******', 'filepath'='mo-big-data/100000000_20_columns_load_data_new.csv'} into table big_data_test.table_basic_for_load_100M fields terminated by '|' lines terminated by '\n' parallel 'true';