Matthew Rocklin

Thanks for joining this conversation @Kirill888 . And thank you for the excellent benchmark. Short term it sounds like there are two easy wins that we might want to do:...

OK, so we should prefer auto-chunking that assumes row-major order among tiles. We might even consider warning is the user asks for something else. Those are pretty easy to achieve....

>> we should prefer auto-chunking that assumes row-major order among tiles > Is this possible currently? Is there a work around, or does xr.open_rasterio need modifying? In this line: with...

A month ago I was doing comparative benchmarks between [Spark/Dask/DuckDB/Polars on cloud data](https://www.youtube.com/watch?v=wKH0-zs2g_U&ab_channel=Coiled). My observations were that, as long as the projects don't do anything dumb (a big assumption) the...

A summary of the finding sounds great. If someone wants to do that that would be welcome.

This notebook might also be of use https://gist.github.com/mrocklin/c1fd89575b40c055a9be77b2a47894df

I personally don't have any thoughts on performance here. Our motivation was to get live figures within a JuptyerLab session so that they could be used alongside a notebook or...

I think that it would be simplest to include an isolated dask-optuna notebook into the examples. I think that that would have good value. It might be useful after that...

@raybellwaves is this something that you would be interested in?

It should be possible to run a dask.distributed client/scheduler/worker in a single thread. Additionally I think that Tornado has a concurrent.futures compatible executor that can be plugged into the worker's...