apache
Introducing Opendal as our new object_store dependency.
Describe This Problem
The introduction of opendal as our object_store dependency has been considered before. We would very much welcome someone from the community who can continue to push this forward.
Proposal
Replace our current object_store dependency with opendal.
Additional Context
No response
Some concerns:
- Will opendal add too many dependencies, if we only use one or two backend?
- How is its performance? especially s3-like backend.
Will opendal add too many dependencies, if we only use one or two backend?
Most of services (storage backend) is optional. I build the default crate (with s3 bundled) and the result is
$ cargo build --release
$ ls -l target/release/libopendal.d
-rw-r--r-- 1 tison staff 11398 Oct 25 06:22 target/release/libopendal.d
How is its performance?
I remember @Xuanwo and @Zheaoli once did a performance test. Maybe they'll have more information.
Most of services (storage backend) is optional.
👍
Then we will wait to see how opendal performed compared with object-store.
Then we will wait to see how opendal performed compared with object-store.
We have added a benchmark inside opendal which compared with aws-sdk-s3, welcome to take a look and bench on your env: https://github.com/apache/incubator-opendal/tree/main/core/benches/vs_s3
Despite potential variations in results, the conclusion is clear: we are all built on rust and powered by hyper, exhibiting similar performance.
