DataProfiler icon indicating copy to clipboard operation
DataProfiler copied to clipboard

Dask Max Version Tag

Open taylorfturner opened this issue 5 months ago • 3 comments

General Information:

  • OS: Linux
  • Python version: 3.9.18
  • Library version: 0.10.9

Describe the bug: Dask change a couple things subsequent to Feb 9, 2024. We have had to put a max pin on Dask version to 2024.2.0. Not pinning outputs the following error in the unit testing suite

____ ERROR collecting dataprofiler/tests/validators/test_base_validators.py ____
/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/dask/dataframe/__init__.py:22: in _dask_expr_enabled
    import dask_expr  # noqa: F401
E   ModuleNotFoundError: No module named 'dask_expr'

During handling of the above exception, another exception occurred:
dataprofiler/tests/validators/test_base_validators.py:4: in <module>
    from dask import dataframe as dd
/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/dask/dataframe/__init__.py:87: in <module>
    if _dask_expr_enabled():
/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/dask/dataframe/__init__.py:24: in _dask_expr_enabled
    raise ValueError("Must install dask-expr to activate query planning.")
E   ValueError: Must install dask-expr to activate query planning.

To Reproduce: Don't pin max on dask in requirements-test.txt file

Expected behavior: No errors -- ideally with no max version pin on dask installation

taylorfturner avatar Mar 22 '24 17:03 taylorfturner

#1120 fixing but ultimately will need to resolve to not have late vintage version pins

taylorfturner avatar Mar 22 '24 19:03 taylorfturner