query_mem_limit is set to certain value even if it is 0 (no limit)

[mcgray]

Seeing:

FileReader::get_next failed. reason = Memory limit exceeded: Memory of Query20875777-6c9d-11ee-b276-0a6694487d1f exceed limit. read and decompress page Used: 4304263376, Limit: 4294967296. Mem usage has exceed the limit of single query, You can change the limit by set session variable query_mem_limit.
/build/starrocks/be/src/formats/parquet/column_chunk_reader.cpp:132 CurrentThread::mem_tracker()->check_mem_limit("read and decompress page")
/build/starrocks/be/src/formats/parquet/column_chunk_reader.cpp:177 _read_and_decompress_page_data(compressed_size, uncompressed_size, true)
/build/starrocks/be/src/formats/parquet/column_chunk_reader.cpp:117 _parse_data_page()
/build/starrocks/be/src/formats/parquet/stored_column_reader.cpp:588 _reader->load_page()
/build/starrocks/be/src/formats/parquet/stored_column_reader.cpp:549 _next_selected_page(records_to_read, content_type, &records_to_skip, dst)
/build/starrocks/be/src/formats/parquet/column_reader.h:72 prepare_batch(num_records, content_type, column), file = s3://fanatics.prod.internal.confidential/orezchykov/iceberg/fanflow_iceberg_db.db/gom_events_test_3m/data/ts_day=2023-01-24/00274-144003-510f0b8f-40b3-470a-b962-3016e61ca861-00001.parquet

Steps to reproduce the behavior (Required)

Running an arbitrary task submit. query_mem_limit is default (which is ZERO) and means "no limit".

Expected behavior (Required)

No limit for query memory

Real behavior (Required)

Limit set to roughly 4GB for 32G EC2 node (which makes me think it is 12.5% or so)

StarRocks version (Required)

3.1.2

[ZiheLiu]

When query_mem_limit is 0, the session variable exec_mem_limit will be effective. The total memory limit of this query in a BE is equal to $exec\_mem\_limit \cdot num\_fragments \cdot pipeline\_dop$, where num_fragments denotes the number of fragments of this query in the BE.

[mcgray]

Can this be added to documentation?

[chaoyli]

@mcgray The exec_mem_limit will be dropped. The query_mem_limit will be the whole memory limit.

[github-actions[bot]]

We have marked this issue as stale because it has been inactive for 6 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to StarRocks!