Charles Stern

@jordanplanders... I'll pull the logs now... (Getting these public by default is one of my top priorities...)

``` fname = config.file_pattern[input_key] File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pangeo_forge_recipes/patterns.py", line 219, in __getitem__ fname = self.format_function(**format_function_kwargs) File "/tmp/tmpzno97m9d/recipes/citrace_tracer/recipe.py", line 33, in make_url NameError: name 'url_d' is not defined [while running 'Start|cache_input|Reshuffle_000|prepare_target|Reshuffle_001|store_chunk|Reshuffle_002|finalize_target|Reshuffle_003/cache_input/Execute-ptransform-56'] ``` Looks...

@jordanplanders, oh sorry didn't realize that `url_d` was defined outside the function scope. This is a known issue with deploying these recipes to Google Cloud Dataflow (where we run them...

No worries about the full variable scope. The cache from this test will be reused for the production run anyway, so we're just getting ahead on data transfer. I'll re-trigger...

/run citrace_tracer

Seeing two different errors on backend now: ``` File "/srv/conda/envs/notebook/lib/python3.9/site-packages/fsspec/asyn.py", line 53, in _runner result[0] = await coro File "/srv/conda/envs/notebook/lib/python3.9/site-packages/fsspec/implementations/http.py", line 420, in _info raise FileNotFoundError(url) from exc FileNotFoundError: https://figshare.com/ndownloader/files/38543969...

@jordanplanders thanks for the follow up. The PR which had been blocking your original feedstock was merged this week. So it's possible we won't need this workaround after all. I've...

This should be as simple as returning a `zarr.storage.FSStore` from this function https://github.com/pangeo-forge/pangeo-forge-recipes/blob/f094b024716485956d81b38c730aee2eaf6ee7da/pangeo_forge_recipes/writers.py#L95-L99 which is what `WriteCombinedReference` calls into here https://github.com/pangeo-forge/pangeo-forge-recipes/blob/f094b024716485956d81b38c730aee2eaf6ee7da/pangeo_forge_recipes/transforms.py#L464-L465

@norlandrhagen, looks like we're quite close but not all the way there: `full_target` is an `pangeo_forge_recipes.storage.FSSpecTarget`. The aim of this issue is to return a `zarr.storage.FSStore`. In practical terms, what...

xref https://github.com/leap-stc/ClimSim/issues/38#issuecomment-1687209517, the slow loading time there can probably be (entirely?) resolved by consolidated metadata.