object-store-python icon indicating copy to clipboard operation
object-store-python copied to clipboard

Async object store support

Open kylebarron opened this issue 1 year ago • 1 comments

Change list

  • Add async methods for each existing function.
  • Remove superfluous vec clone. PyBytes::new(py, &Bytes::to_vec()) had two clones, I believe, where only one was needed.
  • Automatically convert to PyBytes from Cow<[u8]> instead of converting manually. At least for the async version this was a lot easier; I struggled with manually constructing PyBytes objects inside the async closure.

Another option for the API would be to have a separate class AsyncObjectStore that has only the async methods. This might keep the methods cleaner (we wouldn't need to have an async suffix on each method). But alternatively, there's no reason why someone couldn't use the same store for both synchronous and async calls.

This is implemented on top of https://github.com/roeap/object-store-python/pull/5, so ignore the first several commits. It's expected for #5 to be merged first, and then this PR can be squashed.

Closes #2

kylebarron avatar Feb 07 '24 21:02 kylebarron

This looks interesting! Any chance to get this merged and released? @roeap

ion-elgreco avatar May 18 '24 20:05 ion-elgreco

Any progress getting this released?

simonbutt avatar Jun 07 '24 13:06 simonbutt

No progress here. The store API has managed to mature a bit so if someone wanted to come and pick this up (@kylebarron or otherwise), now would be a good time.

jhamman avatar Jun 07 '24 15:06 jhamman

No progress here. The store API has managed to mature a bit so if someone wanted to come and pick this up (@kylebarron or otherwise), now would be a good time.

I have picked it up.

dark0dave avatar Jun 09 '24 11:06 dark0dave

Maybe it's easier to fork, and then merge all open PRs into that fork and release it

ion-elgreco avatar Jun 09 '24 11:06 ion-elgreco