Consider refactoring LruCacheLayer with `list_with_metakey` and `concurrent_stat_in_list`

[WenyXu]

Consider refactoring it with list_with_metakey and concurrent_stat_in_list

Originally posted by @WenyXu in https://github.com/GreptimeTeam/greptimedb/pull/3058#discussion_r1438831103

https://github.com/GreptimeTeam/greptimedb/blob/2f98fa0d97195c4818a9db52032924700ca49577/src/object-store/src/layers/lru_cache/read_cache.rs#L135-L157

[suyanhanx]

I want to try this. Please assign it to me. 😊

[tisonkun]

@suyanhanx Welcome! You're assigned and feel free to leave a comment here to share updates; or ping me as a reviewer if the patch is submitted.

[suyanhanx]

I checked the current implementation and it currently calls mainly on the Accessor. If we want to refactor it with the new methods list_with_metakey and concurrent_stat_in_list, I think we may need to change it to Operator instead. This will remove the trait currently relied upon here and its upstream since Operator is not a trait. https://github.com/GreptimeTeam/greptimedb/blob/2f98fa0d97195c4818a9db52032924700ca49577/src/object-store/src/layers/lru_cache/read_cache.rs#L72-L110

[ozewr]

Hi,Can I give it a try?

[WenyXu]

Hi,Can I give it a try?

Have fun

[ozewr]

Can I replace Access trait to Accessdyntrait?

/opendal/src/raw/accessor.rs:398 /// AccessDyn is the dyn version of [Access] make it possible to use as /// Box<dyn AccessDyn>.

The Operator struct can build from dyn AccessDyn.
This way, I can construct the Operator in the recover_cache function, and then use list_with_metakey and concurrent_stat_in_list.

[WenyXu]

Can I replace Access trait to Accessdyntrait?

/opendal/src/raw/accessor.rs:398 /// AccessDyn is the dyn version of [Access] make it possible to use as /// Box<dyn AccessDyn>.

The Operator struct can build from dyn AccessDyn. This way, I can construct the Operator in the recover_cache function, and then use list_with_metakey and concurrent_stat_in_list.

Yes, I think we should.