new feature: Returning metadata while reading

[Xuanwo]

Feature Description

Most object storage services will return metadata while reading data, it could be great if opendal can support it.

Problem and Solution

Returning metadata while reading data.

Additional Context

OpenDAL used to support it but removed it for handling concurrent reading.

Are you willing to contribute to the development of this feature?

• [ ] Yes, I am willing to contribute to the development of this feature.

[shbhmrzd]

Hi, I would like to work on this. Thank you!

[Xuanwo]

Hi, I would like to work on this. Thank you!

Hi, would you like to propose an RFC first? It will be a big breaking changes for read API.

[shbhmrzd]

Sure, let me work on that first.

[Xuanwo]

Hi, @meteorgan, would you like to update this issue to reflect the latest status?

[meteorgan]

Hi, @meteorgan, would you like to update this issue to reflect the latest status?

I'm afraid we haven't made any progress on this issue so far. @shbhmrzd are you still interested in it ? If not, I can propose an RFC in the next few days.

[Xuanwo]

Oh, I see. I mixed this one with writing return metadata.

[shbhmrzd]

Hi @meteorgan @Xuanwo

Apologies for the delay. I’d love to work on this but can’t dedicate time now due to work commitments. Please feel free to proceed.

[meteorgan]

I'll take on this issue and start by drafting an RFC. I've taken a quick look at the related code and docs, and it looks like this might be a bit of tricky task, we may have a bunch of details to discuss.

[Xuanwo]

Hi, @meteorgan, we can continue the discussion here.

[Xuanwo]

I'm thinking about this for a while. One possible idea is we can store an LazyCell in Reader which can only be filled once after users start a real read.

There are two possible API design:

impl Reader {
    fn metadata(&self) -> Option<&Metadata>;
    async fn metadata(&self) -> &Metadata;
}

for fn metadata(&self) -> Option<&Metadata>:

Reader always return None if we don't know about it, otherwise returns Some(&Metadata).

This one is super clear but might be a bit not useful.

for async fn metadata(&self) -> &Metadata:

Reader can start a stat call internally if the metadata is not know yet.

This one is easy to use but have the following two problems:

• Hidden API call happened.
• stat can return error but there is no way to handle it.