opendal feat: We can buffer some data if user kept writing small chunks

For examples, s3 doesn't support parts that lower than 4MiB, it's cool if we can maintain a small buffer in memory so users don't need to bother.

We can provide a flush API to make sure the buffer is written.

Mar 01 '23 02:03 Xuanwo

Based on the experiment conducted at https://github.com/datafuselabs/opendal/pull/1536 (thanks to @ClSlaid), I have found that adding a flush is not a good idea. Users could call flush multiple times which is unexpected.

Mar 09 '23 10:03 Xuanwo

Maybe we can implement a BytesReader(Vec<Bytes>) so that we don't need to copy data while append?

Mar 09 '23 10:03 Xuanwo

Maybe we can implement a BytesReader(Vec<Bytes>) so that we don't need to copy data while append?

I would really appreciate it if you can elaborate more on this :)

Apr 14 '23 03:04 xyjixyjixyji

I would really appreciate it if you can elaborate more on this :)

Great, I am currently working on it. Shall we meet at the PR?

Apr 14 '23 03:04 Xuanwo

Implemented.

Apr 21 '23 13:04 Xuanwo