Can't import in Google Colab

[randerzander]

Google Colab depends on an older version of prompt-toolkit, breaking dask-sql imports.

!pip install dask-sql

from dask_sql import Context

Trace:

---------------------------------------------------------------------------

ImportError                               Traceback (most recent call last)

<ipython-input-3-68301791ec53> in <module>()
----> 1 from dask_sql import Context
      2 
      3 c = Context()

1 frames

/usr/local/lib/python3.7/dist-packages/dask_sql/cmd.py in <module>()
      6 from dask.datasets import timeseries
      7 from dask.distributed import Client
----> 8 from prompt_toolkit import PromptSession
      9 from prompt_toolkit.lexers import PygmentsLexer
     10 from pygments.lexers.sql import SqlLexer

ImportError: cannot import name 'PromptSession' from 'prompt_toolkit' (/usr/local/lib/python3.7/dist-packages/prompt_toolkit/__init__.py)


---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------

Looking at the dask-sql code, PromptSession is only ever used in cmd.py. If I understand correctly, that's only used as part of the dask-sql command line repl. It may make sense to separate that into its own pip module.

Workaround:

!pip uninstall -y prompt_toolkit
!pip install -U dask-sql prompt-toolkit ipython
** Restart Runtime **

After the above, you can import dask_sql modules as expected.

[nils-braun]

Good catch and thanks for providing already a workaround. And yes, you are completely right: it is only needed for the command line interface. I will check if I might just fix the version dependency. If not, also a lazy import might help.

[nils-braun]

I added a fix in #159 which should make it compatible with older and newer versions of prompt_toolkit. If you want, you can test it following the instructions in the PR :-)

I still like your idea of keeping the CLI separate, but so far I am not 100% sure how it might be implemented best (a really separate package? only as an extra installable?). The PR should at least fix the error and we can come back to this decision after that.

[randerzander]

This issue has resurfaced:

Collecting dask-sql
  Downloading dask_sql-2021.12.0-py3-none-any.whl (19.4 MB)
     |████████████████████████████████| 19.4 MB 3.9 MB/s 
...
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
gym 0.17.3 requires cloudpickle<1.7.0,>=1.2.0, but you have cloudpickle 2.0.0 which is incompatible.
Successfully installed anyio-3.4.0 asgiref-3.4.1 backports.zoneinfo-0.2.1 cloudpickle-2.0.0 dask-2021.11.2 dask-sql-2021.12.0 distributed-2021.11.2 fastapi-0.70.1 fsspec-2021.11.1 h11-0.12.0 jpype1-1.3.0 locket-0.2.1 partd-1.2.0 pydantic-1.8.2 pytz-deprecation-shim-0.1.0.post0 sniffio-1.2.0 starlette-0.16.0 tzdata-2021.5 tzlocal-4.1 uvicorn-0.16.0

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-2-68301791ec53> in <module>()
----> 1 from dask_sql import Context
      2 
      3 c = Context()

1 frames
/usr/local/lib/python3.7/dist-packages/dask_sql/cmd.py in <module>()
     12 from dask.distributed import Client, as_completed
     13 from prompt_toolkit.auto_suggest import AutoSuggestFromHistory
---> 14 from prompt_toolkit.completion import WordCompleter
     15 from prompt_toolkit.history import FileHistory
     16 from prompt_toolkit.shortcuts import ProgressBar

ImportError: cannot import name 'WordCompleter' from 'prompt_toolkit.completion' (/usr/local/lib/python3.7/dist-packages/prompt_toolkit/completion.py)

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.

For what it's worth, Colab's installed version of prompt_toolkit is 1.0.18