distributed
distributed copied to clipboard
dask
A distributed task scheduler for Dask
Hi. I am using client.submit with resources restrictions for root tasks (only CPU only, not GPU) Then these root tasks results (futures) are set as input to subsequent client.submit calls...
Hi, I'm encountering repeated failures during ```xarray.Dataset.to_zarr()``` when using Dask Distributed. The failures occur during the Dask shuffle/rechunk phase and appear to be caused by instability in the P2P shuffle...
**Describe the issue**: I'm seeing high CPU usage (4-6% on an 18 core/36 thread CPU) by "Service Host: DNS Client" associated with dask workers, even when they're not running tasks....
Closes #xxxx - [ ] Tests added / passed - [ ] Passes `pre-commit run --all-files` Closes https://github.com/dask/distributed/issues/2007 since other classes have been implemented. Happy to get feedback on this.
Added some type hinting for `distributed.client.FutureState.status` and the `distributed.client.Future.status` property that returns it. Updated the docstring of `Future.status` to also include these states. The type hint has to include `None`...
A `Future` has a `status` property. The [current documentation](https://distributed.dask.org/en/latest/api.html#distributed.Future.status) only says `Returns the status` and that the output is a string. The definition of this is a property which returns...
**Describe the issue**: I'm processing NetCDF and converting them to Zarr with xarray. For this, I'm using a coiled cluster, dask, xarray, s3fs... As a normal user who just wants...
[Polars](https://github.com/pola-rs/polars) is a single-machine columnar dataframe library built on top of pyarrow Tables with both a eager and lazy API. While the default pickle serialization works fine for eagerly-created dataframes,...
Dask currently has some distributed coordination primitives like Locks and Queues that match the API of normal local Locks and Queues. There are other primitives like these that we do...
**Describe the issue**: **Minimal Complete Verifiable Example**: This is broken for both sync and async delayed functions. Calling functions directly (removing delayed decorator) preserves spans and annotations. This is an...
