webknossos icon indicating copy to clipboard operation
webknossos copied to clipboard

Zarr backend follow-up: tech

Open fm3 opened this issue 2 years ago • 1 comments

Follow-up for reading zarr-datasets in wk backend:

  • [ ] cache clear for file system handles (when password for an account changes / file system breaks somehow)
  • [ ] double-check that fill value is used if chunk does not exist
  • [ ] support string-encoded fill values
  • [x] perf: async cache, for file systems, handles – use akka-caching (caution: Futures all around, ~needs actorSystem~)
  • [x] do not depend on jzarr, instead on its dependencies itself (only those we use) https://github.com/scalableminds/webknossos/pull/6152
  • [ ] better error messages when bbox is not right (e.g. assertions in computeChunkIndices)
  • [ ] where to store file systems / file system provider ref
  • [x] perf optimization: skip allocating buffer if single-bucket request
  • [ ] perf optimization: if single-bucket request, skip typing of buffers

fm3 avatar Mar 22 '22 06:03 fm3

I’d say with #6152 and #6165 the two most important parts of this issue are taken care of. Let’s schedule it again when we feel that the other checkboxes are relevant enough :)

fm3 avatar Apr 26 '22 12:04 fm3

I split the remaining parts of this issue into #6989 and #6990

fm3 avatar Apr 18 '23 13:04 fm3