Charles Stern

Finding a solution here is still worthwhile IMO, though will presumably unfold over a somewhat longer timescale. In the meantime, please to report that https://github.com/dask/distributed/pull/8400 does unblock many use cases,...

> https://github.com/dask/distributed/pull/8400 does unblock many use cases, including the linked Beam issue I spoke a bit too soon here. As described at the bottom of https://github.com/apache/beam/pull/29802#issuecomment-1863683005, while https://github.com/dask/distributed/pull/8400 did resolve...

Ping for those following here, I've just opened my proposed fix for this issue, https://github.com/dask/dask/pull/10734, for review.

Here's a way to sanitize en-dashes: ```python "MPI‐M".encode("utf-8").replace(b"\xe2\x80\x90", b"-").decode() ```

(Deleted last comment because I realized you were saying that there are _two_ forms of the dash that use surrogates!)

Possibly related to https://github.com/pangeo-forge/pangeo-forge-runner/issues/78?

Thanks for opening this, @cmdupuis3! > it could also make sense to break it into two separate datasets with a simpler structure, obviating the need for a tree-like structure This...

> Quick question for now: why not use Beam’s built-in streaming primitives? The KISS solution here, IMO, is to build off of Beam’s PubSub and Kafka connectors to react to...

Thanks for raising this, @abarciauskas-bgse. Based on the notebook it looks like you are using a development version of pangeo-forge-recipes that predates the 0.10.0 release? I don't have any specific...

Thanks for checking that, @abarciauskas-bgse. AFAICT, this feature did exist in `0.9.4` here: https://github.com/pangeo-forge/pangeo-forge-recipes/blob/3b3c13c358259a9b01367c91951d28826b6c252e/pangeo_forge_recipes/recipes/xarray_zarr.py#L665-L692 And re-implementing this as part of the Beam rewrite was simply overlooked. Adding this would probably...