Xuanwo
Xuanwo
Current failling tests: https://github.com/apache/opendal/issues/5801
Thank you!
Hi, OpenDAL handles those connection-related tasks (by reqwest). Currently, FileIO builds new operators every time: https://github.com/apache/iceberg-rust/blob/4083f8109b1059c6e3cb7df7ae50a2c8cdc8a613/crates/iceberg/src/io/storage.rs#L104-L119 I think we can improve this by using the same HTTP client instead.
Thanks for the sharing. I believe this can be addressed by reusing the same http client.
We will need https://github.com/apache/opendal/issues/4967 for this. I'm working on it now.
> Which opendal's version do we need to bump into? I'm guessing it will be included in our next release `0.49`.
Also, cc @sdd, who is focusing on the iceberg benchmark now.
> I also have some local code that reuses the same OpenDAL operator rather than creating a new one each time. I'd not submitted it yet as I wasn't sure...
Let me take this one.
Interesting. I'm not very familiar with how Docker volumes work. It seems we can implement such a volume plugin, but I'm unsure about the IO processes involved. I assume we...