pangeo-forge-recipes
pangeo-forge-recipes copied to clipboard
Disambiguate naming of `store_chunk`/`iter_chunks` from `target_chunks`
XarrayZarrRecipe
uses variations of the name chunks
to refer to two different recipe attributes. iter_chunks
is a generator which yields inputs for the store_chunk
execution step. target_chunks
is a dictionary defining zarr chunk sizes. It may be worth considering re-naming iter_chunks
(and any/all associated recipe attributes) to make it clear that this method is not related to zarr chunks.
Possible alternate names
- fragments
- shards
- blocks
- segments
- write fragments / blocks / shards / segments
fragments shards blocks segments
shards or blocks are appealing to me simply for being shorter, and between them, shards seems less connotative of byte sequences within a file. so maybe shards?
Would like to add an endorsement for following the target_chunks
naming pattern (<adjective>_<noun>
) in the renaming of store_chunk
/iter_chunks
. Naming in storage.py defines a source
/target
relationship, so perhaps store_chunk
could become something like ... write_source_shard
? Which would make iter_chunks
... iter_source_shards
.
For anyone looking to grok the somewhat hard-to-grasp notion of the "source chunks" (under discussion for renaming here), reading https://github.com/pangeo-forge/pangeo-forge-recipes/pull/205#issuecomment-923200879 was an "a-ha" moment for me.
This issue relates to the pre-beam-refactor XarrayZarrRecipe
which we are no longer developing, so closing. As an aside, I think the current StoreToZarr
naming does a good job of addressing this prior naming overload, by distinguishing between target_chunks
(i.e. zarr chunks), and fragments
(which are used to create zarr chunks, but are not 1:1 with zarr chunks).