feat: allow using object_store as opendal's backend

Open flaneur2020 opened this issue 6 months ago • 2 comments

Which issue does this PR close?

Closes #6171.

Rationale for this change

with allowing object_store as opendal's backend, we can leverage opendal's advanced operation like parallel fetching, chunking, caching for users object_store.

this pr is still in wip

What changes are included in this PR?

Are there any user-facing changes?

Jun 11 '25 13:06 flaneur2020

Thanks a lot for building this, but I think it's better to be part of opendal's object_store integration instead of in opendal core.

Jun 11 '25 16:06 Xuanwo

Thanks a lot for building this, but I think it's better to be part of opendal's object_store integration instead of in opendal core.

thank you for the feedback, let me move it there

Jun 12 '25 04:06 flaneur2020

Give Github Copilot a chance.

Jul 04 '25 10:07 Xuanwo

Thank you for this! I will start review next week (this week is bit busy)

Jul 24 '25 14:07 Xuanwo

Thank you for this! I will start review next week (this week is bit busy)

no hurry at all! take your time, looking forward to your feedback whenever it's convenient for you :)

Jul 25 '25 05:07 flaneur2020

@Xuanwo sorry for the late response. i updated the PR with following the feedbacks. please take another look when you have time at your convenience qwq

there're still some follow ups that might be in future prs:

make it integrated with opendal's existing test suites
enhance the metadata handling
make concurrent configurable in writer

Aug 16 '25 04:08 flaneur2020