Jeremy Maitin-Shepard

This would be really valuable! Do you plan to keep working on this?

Currently listing for neuroglancer_uint64_sharded is not optimized --- internally it is only used for unit tests so it just reads all of the keys (which requires reading all minishard indices)...

There is still some underlying bug, though --- it shouldn't be necessary to remove those icons. Can you share a minimal reproduction?

> Ok I think I found the issues! > > 1. It seems TensorStore expects the [first byte of the generation to be 0](https://github.com/google/tensorstore/blob/933712f0628f9170ed00eafdd561402eb6fb77fe/tensorstore/kvstore/generation.cc#L221), but the [gRPC driver just directly...

This is great --- I haven't had a chance to review your implementation in detail, but at a high level this is exactly the right implementation strategy. Do you have...

Regarding logging: what have you tried? Are you testing with bazel? With bazel you can do `--test_env=TENSORSTORE_VERBOSE_LOGGING=tiff`

You have made some good observations --- kvs_backed_chunk_driver indeed currently assumes a zarr-like structure. Ultimately it could be nice if we refactored things so that some of the logic of...

There isn't a format supported by tensorstore that works exactly like that, but the OCDBT format (https://google.github.io/tensorstore/kvstore/ocdbt/index.html) does something similar and might suit your needs.

Is there a reason that you can't store those two fields as separate arrays? Tensorstore supports this for zarr v2 but exposes each field as a separate array. For zarr...

Regarding your question of whether tensorstore copies the data: Tensorstore copies into the chunk cache by default; in this case once in the chunk cache, they can be written directly...