feat(storage): split duties of sstable_store `load_table` (load blocks) and `sstable` (meta only), disambiguate `TableHolder::{CachedMeta, Owned}`, do not store blocks in `meta_cache`

[jon-chuang]

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

What's changed and what's your intention?

Remove abuse of sstable_store meta_cache to store data blocks.

Make it slighly less confusing whether TableHolder ought to contain blocks.

Todo:

1. We will validate via bench on S3 whether with this change, we will no longer see high memory usage in compactor. a. (on second thought, I think the high memory usage bug does not really lie here as the meta_cache's capacity is already limited to 1GB. However, note that duplicated requests to the sst_id will result in those requesters holding a reference to a sstable with blocks, despite only requesting for sstable meta. Perhaps this could be a source of memory leak? Answer: no, if CachableEntry is referenced, it will still take up usage in the cache, so such a leak will show up in the compactor cache metrics.)
2. can we get consensus to rename load_table and sstable to load_table_with_blocks and load_table_meta respectively?

Refer to a related PR or issue link (optional)

https://github.com/singularity-data/risingwave/issues/3921 Details: https://github.com/singularity-data/risingwave/issues/3921#issuecomment-1193126155

[codecov[bot]]

Codecov Report

Merging #4123 (0097bc6) into main (093addf) will increase coverage by 0.00%. The diff coverage is 84.84%.

@@           Coverage Diff           @@
##             main    #4123   +/-   ##
=======================================
  Coverage   73.95%   73.95%           
=======================================
  Files         842      842           
  Lines      119473   119485   +12     
=======================================
+ Hits        88356    88367   +11     
- Misses      31117    31118    +1     

Flag	Coverage Δ
rust	73.95% <84.84%> (+<0.01%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/ctl/src/cmd_impl/hummock/sst_dump.rs	0.00% <0.00%> (ø)
src/storage/src/hummock/iterator/concat_inner.rs	95.18% <0.00%> (ø)
src/storage/src/hummock/sstable_store.rs	79.68% <83.67%> (-0.97%)	:arrow_down:
src/storage/src/hummock/compactor.rs	78.69% <100.00%> (ø)
...e/src/hummock/sstable/backward_sstable_iterator.rs	97.53% <100.00%> (+0.07%)	:arrow_up:
...ge/src/hummock/sstable/forward_sstable_iterator.rs	95.90% <100.00%> (+0.07%)	:arrow_up:
src/meta/src/manager/id.rs	94.94% <0.00%> (-0.57%)	:arrow_down:
src/storage/src/hummock/local_version_manager.rs	74.65% <0.00%> (+0.15%)	:arrow_up:
src/meta/src/hummock/mock_hummock_meta_client.rs	40.86% <0.00%> (+0.86%)	:arrow_up:
... and 1 more

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

[github-actions[bot]]

This PR has been open for 60 days with no activity. Could you please update the status? Feel free to ping a reviewer if you are waiting for review.

[lmatz]

Is Remove abuse of sstable_store meta_cache to store data blocks. an already solved issue? If yes, we may close the PR