doris icon indicating copy to clipboard operation
doris copied to clipboard

[Bug] Not enough memory to create materialized view

Open Jrebel-i opened this issue 9 months ago • 0 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

Version

2.0.7

What's Wrong?

When I create the materialized view, I get the following error:

Msg: errCode = 2, detailMessage = rollup tasks failed on same tablet reach threshold [[ALTER], signature: 1116543, backendId: 10642, tablet id: 1116543], reason=task type: ALTER, status_code: MEM_ALLOC_FAILED, status_message: [(10.150.31.199)[MEM_ALLOC_FAILED][E11] Allocator mem tracker check failed, [MEM_LIMIT_EXCEEDED]failed alloc size 8.00 KB, memory tracker limit exceeded, tracker label:EngineAlterTabletTask#baseTabletId=829115:newTabletId=1116543, type:schema_change, limit 2.00 GB, peak used 2.00 GB, current used 2.00 GB. backend 10.150.31.199 process memory used 7.63 GB. can modify `memory_limitation_per_thread_for_schema_change_bytes` in be.conf to change limit, details see be.INFO

Then I increase the memory_limitation_per_thread_for_schema_change_bytes=51539607552 for all be nodes and report the following error again:

Msg: errCode = 2, detailMessage = rollup tasks failed on same tablet reach threshold [[ALTER], signature: 1255382, backendId: 10642, tablet id: 1255382], reason=task type: ALTER, status_code: MEM_LIMIT_EXCEEDED, status_message: [(10.150.31.199)[MEM_LIMIT_EXCEEDED]PreCatch error code:11, [E11] Allocator sys memory check failed: Cannot alloc:262144, consuming tracker:<EngineAlterTabletTask#baseTabletId=860575:newTabletId=1255382>, peak used 17571329400, current used 17571329400, exec node:<>, process memory used 49.28 GB exceed limit 49.43 GB or sys available memory 1.41 GB less than low water mark 1.60 GB.
The maximum memory of the server is 64 GB.

What You Expected?

Create the materialized view correctly

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

Jrebel-i avatar Apr 30 '24 06:04 Jrebel-i