sourmash icon indicating copy to clipboard operation
sourmash copied to clipboard

explore extending branchwater plugin to load zip files for prefetch and gather, entirely on rust side?

Open ctb opened this issue 1 month ago • 1 comments

this could be a good & substantive next step towards higher-level oxidation of sourmash internals - what if we added a load_from plugin entry point to branchwater that loaded zip files, and supported the Index API / passed the index API tests?

this could potentially then support multithreaded prefetch, and hence faster gather.

it would take a while to work, but we could do it incrementally, I suspect...

related issues:

  • https://github.com/sourmash-bio/sourmash/issues/1939

ctb avatar Jun 29 '24 19:06 ctb