datafuselabs
Feature: allow user to query on spilled storage size on system.query_log table
Summary
Description for this feature.
Consider to add two fields
bytes_spilled_to_local_storage and bytes_spilled_to_remote_storage to system.query_log table
allow user to checkout the number of bytes spilled to local storage and remote storage could help them troubleshooting their query performance issue when query consume too much time.
Ref: https://docs.snowflake.com/en/user-guide/performance-query-warehouse-memory
Are the fields bytes_spilled_to_local_storage and bytes_spilled_to_remote_storage the same as bytes_from_local_disk and bytes_from_remote_disk? If so, why add these two redundant fields?
Are the fields
bytes_spilled_to_local_storageandbytes_spilled_to_remote_storagethe same asbytes_from_local_diskandbytes_from_remote_disk? If so, why add these two redundant fields?
After a spill, the read may not occur or it could take place on a different query node. Therefore, we need to determine the number of spilled bytes.
And it is primarily serve for estimating proper warehouse size for curtain query category, if customer found that the spilled bytes is pretty high, they could consider to upgrade warehouse to save query time.
