rust-oci-client icon indicating copy to clipboard operation
rust-oci-client copied to clipboard

Provide seek-able interface to Client

Open thundergolfer opened this issue 1 year ago • 2 comments

rust-oci-client has Client::pull_blob_stream but this is not seek-able. I'm looking for a seek-able interface because it allows me to read ranges of bytes from a blob and extract a Toc (Table-of-Contents) data structure from an eStargz image.

stargz-snapshotter uses containerd's seek-able ContentStore to achieve the same.

We at modal.com are open to upstreaming this functionality if suitable :)

thundergolfer avatar Sep 03 '24 18:09 thundergolfer

Thanks for the issue @thundergolfer! Is there any reason what you're doing couldn't be achieved with using StreamExt (or TryStreamExt) methods like chunks or take?

Using something like AsyncSeek wouldn't work unless we buffered into memory (because we can't go backwards in the stream)

thomastaylor312 avatar Sep 04 '24 22:09 thomastaylor312

Thanks for the reply @thomastaylor312. I think I ultimately need an interface which will materialize arbitrary HTTP range requests against the blob and though I don't get how chunks would do it I think skip + take may end up working.

I do see now that pull_blob_response accepts an offset which gets stuck into a range header. So if there was just an additional len: u64 or similar to do bytes={offset}-{offset+len} then I think I'd be done :)

thundergolfer avatar Sep 05 '24 13:09 thundergolfer

Closed in #163

thomastaylor312 avatar Oct 04 '24 20:10 thomastaylor312