Tom White
Tom White
#291 added batching for Python Async, Modal, and Dask.
The code I used to produce the above plot is in [this branch](https://github.com/tomwhite/cubed/tree/memray), in particular [this commit](https://github.com/tomwhite/cubed/commit/bc51fc93fb246978248ddb7a52a4563c12e7cbab). It would be be good to have more control over i) the file...
Zarr 3.0.0a0 was released last week. It's on [PyPI](https://pypi.org/project/zarr/3.0.0a0/), and can be installed by specifying `--pre` with `pip install`. There's already a GH Actions workflow for this, which is currently...
[Jacob Tomlinson on twitter](https://twitter.com/_JacobTomlinson/status/1675077014716391425) suggested [momento](https://www.gomomento.com/), [Dragonfly](https://www.dragonflydb.io/), and [ElastiCache](https://aws.amazon.com/elasticache/) as possibilities too.
Also relevant to this issue is the work going on in Zarr to improve performance. * Discussion thread: https://github.com/zarr-developers/zarr-python/discussions/1479 * Cubed use case (by @TomNicholas): https://github.com/zarr-developers/zarr-python/discussions/1486#discussioncomment-6732054
> * Is fsspec + s3fs + Zarr likely to "just work" with this new storage class? It says it implements most of the same S3 API... Not yet, by...
> Let's just create python bindings for Rust's [object store](https://docs.rs/object_store/latest/object_store/) crate and start using that instead. Looks very powerful and performant. More performance would be welcome! It doesn't support S3...
Thanks for all the comments! Really interesting. > I starting pushing on Arrow timings (see [apache/arrow#38389](https://github.com/apache/arrow/issues/38389)) and found that it has much less to do with the language that is...
When doing #309 I noticed that the underlying Dask client used for `coiled.function` is *not* asynchronous. So to do this PR we'd need to get the `cluster` object from the...
> * Would it make sense to name a task corresponding to array creation using the corresponding xarray `DataArray.name`? Other downstream tasks are composed from multiple arrays, so it's not...