[BUG] building a wheel on windows with python 3.11 fails

[markope]

When I try to build a wheel on python 3.11 on my windows2012 R2 Server I get an error related to setuptools_rust. I do have setuptools and setuptools_rust installed on my python. Verified on the command line python with

import setuptools_rust

With this setup when I execute

python -m pip wheel dask_sql

then I get this output:

What happened:

PS C:\pypi> py -m pip wheel dask_sql
Collecting dask_sql
  Using cached dask_sql-2023.4.0.tar.gz (193 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ?─> [21 lines of output]
      Traceback (most recent call last):
        File "C:\Users\Administrator\AppData\Local\Programs\Python\Python311\pip\_vendor\pyproject_hooks\_in_process\_in
_process.py", line 353, in <module>
          main()
        File "C:\Users\Administrator\AppData\Local\Programs\Python\Python311\pip\_vendor\pyproject_hooks\_in_process\_in
_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\Administrator\AppData\Local\Programs\Python\Python311\pip\_vendor\pyproject_hooks\_in_process\_in
_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\Administrator\AppData\Local\Temp\pip-build-env-uycquv7l\overlay\Lib\site-packages\setuptools\buil
d_meta.py", line 341, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\Administrator\AppData\Local\Temp\pip-build-env-uycquv7l\overlay\Lib\site-packages\setuptools\buil
d_meta.py", line 323, in _get_build_requires
          self.run_setup()
        File "C:\Users\Administrator\AppData\Local\Temp\pip-build-env-uycquv7l\overlay\Lib\site-packages\setuptools\buil
d_meta.py", line 488, in run_setup
          self).run_setup(setup_script=setup_script)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\Administrator\AppData\Local\Temp\pip-build-env-uycquv7l\overlay\Lib\site-packages\setuptools\buil
d_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 5, in <module>
      ModuleNotFoundError: No module named 'setuptools_rust'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
?─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

What you expected to happen:

Either a wheel already exists or I am able to build a wheel myself. ( On pypi.org I see that wheels are only provided for python up tp 3.10 )

Environment:

• dask-sql version: 2023.4.0
• Python version: 3.11
• Operating System: Windows Server 2012R2 (64bit)
• Install method (conda, pip, source): pip build from wheel

[charlesbluca]

I'm not able to reproduce these failures locally - could you give a list of steps you went through to initialize your environment? In particular, I ran:

$ conda create -n test-dask-sql-py311-wheel python=3.11 setuptools-rust
$ conda activate test-dask-sql-py311-wheel
$ python -m pip wheel dask_sql

Relevant to this issue is #1143, where I started evaluating the blockers to introducing support for 3.11; as of now, we're waiting for 3.11 builds of mlflow to be published.

[markope]

1. I am working in a fresh VM with Windows Server 2012R2

2. I installed the required Microsoft Visual C++ Redistributable

3. I installed python using the official windows installer from python.org https://www.python.org/ftp/python/3.11.3/python-3.11.3-amd64.exe

4. No conda involved. I also did not create a virtualenv.

5. I did install setuptools and setuptools_rust with python -m pip install setuptools setuptools_rust

6. I run python -m pip wheel das_sql Then the described error appears.

---

As a temporary workaround I use dask-sql==2022.08.0 there the command works:

python -m pip wheel dask_sql

[charlesbluca]

Tried to roughly recreate this process by pulling in a Docker image running Python 3.11 on Windows Server:

docker run --rm -it --entrypoint cmd python:3.11-windowsservercore-ltsc2022

And running the following commands:

cd Users\Default
python -m pip install setuptools_rust
python -m pip wheel dask_sql

Doing this, I was unable to reproduce the setuptools_rust error you encountered; I suspect this could be an issue with the specific Python installation method you're using, some relevant things to know here are:

• what method you're using to spin up the Windows VM, in particular if it comes with some version of python pre-installed
• what the output of python -m pip list looks like immediately after installing setuptools_rust

If the main concern here is availability of dask-sql wheels on python 3.11, there is currently work being done in #1143 to begin publishing 3.11 wheels.