spatialdata icon indicating copy to clipboard operation
spatialdata copied to clipboard

v0.2.6 broken due to updates in Dask

Open adkinsrs opened this issue 11 months ago • 4 comments

NOTE: This can be easily solved by updating to v0.2.7-pre0, but I thought I would notify of the breaking change anyways.

Within the past hour of me posting this, the "dask" package posted a new 2025.1.0 release that removes some legacy dask support. In spatialdata==0.2.6 there is no specific version of dask specified in the pyproject.toml dependencies. If you try to run import spatialdata as sd it will error out with a message like "The legacy implementation is no longer supported" referring to dask.dataframe

This was fixed in the v0.2.7-pre0 tag, but I still think it would be a good idea to either have a notice somewhere that 0.2.6 will not work or backport the Dask dependency settings from the current pyproject.toml to v0.2.6.

adkinsrs avatar Jan 17 '25 18:01 adkinsrs

Thank you for reporting; we are going to release 0.2.7 soon, which as you observed will include the constraint for Dask.

LucaMarconato avatar Jan 20 '25 13:01 LucaMarconato

Same is happening to me, which is breaking all tests of a tool I am developing, and uses squidpy (therefore spatialdata too) as a requirement:

from spatialdata import SpatialData
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/spatialdata/__init__.py:6: in <module>
    import dask.dataframe as dd
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/dask/dataframe/__init__.py:14: in <module>
    _dask_expr_enabled()
/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/dask/dataframe/__init__.py:9: in _dask_expr_enabled
    raise NotImplementedError("The legacy implementation is no longer supported")

earmingol avatar Jan 23 '25 10:01 earmingol

@earmingol I have made a release 3 days ago which adds a constraint for Dask when installing spatialdata. Unfortunately there is no easy solution for this for the moment, and therefore one would need to downgrade Dask.

The problem is due to the fact that dask-expr, which is the new default backend for dask.dataframe.DataFrame dropped .attrs. Without that, we cannot represent points elements: https://github.com/dask/dask/issues/11146 https://github.com/pandas-dev/pandas/issues/52166.

LucaMarconato avatar Jan 23 '25 15:01 LucaMarconato

You can install Dask==2024.12.1 to your environment, which does seem to work with spatialdata 0.2.6

adkinsrs avatar Jan 23 '25 15:01 adkinsrs