python-stratify icon indicating copy to clipboard operation
python-stratify copied to clipboard

Dask API change triggers AttributeError in vinterp

Open valeriupredoi opened this issue 6 months ago • 3 comments

hey folks, we're seeing an issue with the current main here, and Dask (2025.3 and 2025.5), see below:

src/stratify/_vinterp.pyx:551: in stratify._vinterp.interpolate
    ???
/opt/conda/envs/esmvaltool/lib/python3.13/site-packages/dask/array/core.py:905: in map_blocks
    out = blockwise(
/opt/conda/envs/esmvaltool/lib/python3.13/site-packages/dask/array/blockwise.py:249: in blockwise
    base.tokenize(
/opt/conda/envs/esmvaltool/lib/python3.13/site-packages/dask/tokenize.py:76: in tokenize
    return _tokenize(*args, **kwargs)
/opt/conda/envs/esmvaltool/lib/python3.13/site-packages/dask/tokenize.py:34: in _tokenize
    token: object = _normalize_seq_func(args)
/opt/conda/envs/esmvaltool/lib/python3.13/site-packages/dask/tokenize.py:154: in _normalize_seq_func
    return tuple(map(_inner_normalize_token, seq))
/opt/conda/envs/esmvaltool/lib/python3.13/site-packages/dask/tokenize.py:147: in _inner_normalize_token
    return normalize_token(item)
/opt/conda/envs/esmvaltool/lib/python3.13/site-packages/dask/utils.py:778: in __call__
    return meth(arg, *args, **kwargs)
/opt/conda/envs/esmvaltool/lib/python3.13/site-packages/dask/tokenize.py:176: in normalize_partial
    return _normalize_seq_func((func.func, func.args, func.keywords))
/opt/conda/envs/esmvaltool/lib/python3.13/site-packages/dask/tokenize.py:154: in _normalize_seq_func
    return tuple(map(_inner_normalize_token, seq))
/opt/conda/envs/esmvaltool/lib/python3.13/site-packages/dask/tokenize.py:147: in _inner_normalize_token
    return normalize_token(item)
/opt/conda/envs/esmvaltool/lib/python3.13/site-packages/dask/utils.py:777: in __call__
    meth = self.dispatch(type(arg))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dask.utils.Dispatch object at 0x7f14fa99b610>
cls = <class '_cython_3_1_0.cython_function_or_method'>

    def dispatch(self, cls):
        """Return the function implementation for the given ``cls``"""
        lk = self._lookup
        if cls in lk:
            return lk[cls]
        for cls2 in cls.__mro__:
            # Is a lazy registration function present?
>           toplevel, _, _ = cls2.__module__.partition(".")
E           AttributeError: 'member_descriptor' object has no attribute 'partition'

/opt/conda/envs/esmvaltool/lib/python3.13/site-packages/dask/utils.py:750: AttributeError

Full stack trace and environment

Thanks to @schlunma for spotting this! Please close and my apologies if you are already aware of this 🍻

valeriupredoi avatar May 14 '25 14:05 valeriupredoi

From @SciTools/peloton: we have been looking at this in great detail. Cython is the prime suspect 🕵️‍♀️, but there is more to investigate first

trexfeathers avatar May 28 '25 09:05 trexfeathers

very many thanks as ever @trexfeathers - if it's Cython then this is something that moves through Python versions, since we've recently started supporting Python 3.13 (as of yesterday eh), and we still see the bugger 🍺

valeriupredoi avatar May 28 '25 13:05 valeriupredoi

We have identified an incompatibility between Cython versions since 3.1.0 and Dask. When the Cython version is <3.1.0 the problem goes away. So we have added that pin.

We found dask/dask#11974. When using this code change the problem goes away. So when we have a new release of Dask we will be able to remove the pin and it should work.

HGWright avatar Jun 09 '25 14:06 HGWright