toga
toga copied to clipboard
beeware
Running Testbed fails on Python 3.12 due to old Pillow version
Describe the bug
Trying to do briefcase dev --test will result in briefcase first installing the requirements which will fail due to missing zlib header files. However, this error is not encountered when installing the core, dummy and winforms backend using:
pip install -e ./core[dev] -e ./dummy -e ./winforms
But, the requirement always fail to install when running briefcase dev --test
I think this is related to the current integration with Pillow.
Steps to reproduce
- Clone the latest main repo
- Install with
pip install -e ./core[dev] -e ./dummy -e ./winforms. It will install successfully. - Go to
testbeddirectory and runbriefcase dev --test - See error:
PS E:\patch-20\testbed> briefcase dev --test
[testbed] Installing requirements...
Processing e:\patch-20\core
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Processing e:\patch-20\winforms
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting coverage==7.2.0
Using cached coverage-7.2.0-cp312-cp312-win_amd64.whl
Collecting pytest==7.2.0
Using cached pytest-7.2.0-py3-none-any.whl (316 kB)
Collecting pytest-asyncio==0.20.3
Using cached pytest_asyncio-0.20.3-py3-none-any.whl (12 kB)
Collecting pillow==9.2.0
Using cached Pillow-9.2.0.tar.gz (50.0 MB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting attrs>=19.2.0 (from pytest==7.2.0)
Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Requirement already satisfied: iniconfig in c:\users\proneon267\appdata\local\programs\python\python312\lib\site-packages (from pytest==7.2.0) (2.0.0)
Requirement already satisfied: packaging in c:\users\proneon267\appdata\local\programs\python\python312\lib\site-packages (from pytest==7.2.0) (23.2)
Requirement already satisfied: pluggy<2.0,>=0.12 in c:\users\proneon267\appdata\local\programs\python\python312\lib\site-packages (from pytest==7.2.0) (1.3.0)
Requirement already satisfied: colorama in c:\users\proneon267\appdata\local\programs\python\python312\lib\site-packages (from pytest==7.2.0) (0.4.6)
Requirement already satisfied: travertino>=0.3.0 in c:\users\proneon267\appdata\local\programs\python\python312\lib\site-packages (from toga-core==0.1.dev7857+g2e16e16) (0.3.0)
Requirement already satisfied: pythonnet>=3.0.0 in c:\users\proneon267\appdata\local\programs\python\python312\lib\site-packages (from toga-winforms==0.1.dev7857+g2e16e16) (3.0.3)
Requirement already satisfied: clr-loader<0.3.0,>=0.2.6 in c:\users\proneon267\appdata\local\programs\python\python312\lib\site-packages (from pythonnet>=3.0.0->toga-winforms==0.1.dev7857+g2e16e16) (0.2.6)
Requirement already satisfied: cffi>=1.13 in c:\users\proneon267\appdata\local\programs\python\python312\lib\site-packages (from clr-loader<0.3.0,>=0.2.6->pythonnet>=3.0.0->toga-winforms==0.1.dev7857+g2e16e16) (1.16.0)
Requirement already satisfied: pycparser in c:\users\proneon267\appdata\local\programs\python\python312\lib\site-packages (from cffi>=1.13->clr-loader<0.3.0,>=0.2.6->pythonnet>=3.0.0->toga-winforms==0.1.dev7857+g2e16e16) (2.21)
Building wheels for collected packages: pillow, toga-core, toga-winforms
Building wheel for pillow (setup.py): started
Building wheel for pillow (setup.py): finished with status 'error'
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [193 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-312
creating build\lib.win-amd64-cpython-312\PIL
copying src\PIL\BdfFontFile.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\BlpImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\BmpImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\BufrStubImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ContainerIO.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\CurImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\DcxImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\DdsImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\EpsImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ExifTags.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\features.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\FitsImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\FitsStubImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\FliImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\FontFile.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\FpxImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\FtexImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\GbrImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\GdImageFile.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\GifImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\GimpGradientFile.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\GimpPaletteFile.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\GribStubImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\Hdf5StubImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\IcnsImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\IcoImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\Image.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageChops.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageCms.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageColor.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageDraw.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageDraw2.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageEnhance.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageFile.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageFilter.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageFont.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageGrab.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageMath.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageMode.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageMorph.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageOps.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImagePalette.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImagePath.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageQt.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageSequence.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageShow.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageStat.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageTk.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageTransform.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImageWin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\ImtImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\IptcImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\Jpeg2KImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\JpegImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\JpegPresets.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\McIdasImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\MicImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\MpegImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\MpoImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\MspImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\PaletteFile.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\PalmImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\PcdImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\PcfFontFile.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\PcxImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\PdfImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\PdfParser.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\PixarImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\PngImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\PpmImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\PsdImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\PSDraw.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\PyAccess.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\SgiImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\SpiderImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\SunImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\TarIO.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\TgaImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\TiffImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\TiffTags.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\WalImageFile.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\WebPImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\WmfImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\XbmImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\XpmImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\XVThumbImagePlugin.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\_binary.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\_deprecate.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\_tkinter_finder.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\_util.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\_version.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\__init__.py -> build\lib.win-amd64-cpython-312\PIL
copying src\PIL\__main__.py -> build\lib.win-amd64-cpython-312\PIL
running egg_info
writing src\Pillow.egg-info\PKG-INFO
writing dependency_links to src\Pillow.egg-info\dependency_links.txt
writing requirements to src\Pillow.egg-info\requires.txt
writing top-level names to src\Pillow.egg-info\top_level.txt
[12/22/23 03:58:35] ERROR listing git files failed - pretending git.py:24
there aren't any
reading manifest file 'src\Pillow.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.c'
warning: no files found matching '*.h'
warning: no files found matching '*.sh'
warning: no files found matching '*.txt'
warning: no previously-included files found matching '.appveyor.yml'
warning: no previously-included files found matching '.clang-format'
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching '.editorconfig'
warning: no previously-included files found matching '.readthedocs.yml'
warning: no previously-included files found matching 'codecov.yml'
warning: no previously-included files matching '.git*' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
no previously-included directories found matching '.ci'
adding license file 'LICENSE'
writing manifest file 'src\Pillow.egg-info\SOURCES.txt'
running build_ext
The headers or library files could not be found for zlib,
a required dependency when compiling Pillow from source.
Please see the install instructions at:
https://pillow.readthedocs.io/en/latest/installation.html
Traceback (most recent call last):
File "C:\Users\proneon267\AppData\Local\Temp\pip-install-tn4ge2dw\pillow_3e4dcb9162ff4e10a8de3cb36afc200c\setup.py", line 995, in <module>
setup(
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\__init__.py", line 103, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
dist.run_commands()
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
self.run_command(cmd)
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\dist.py", line 963, in run_command
super().run_command(command)
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
cmd_obj.run()
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\wheel\bdist_wheel.py", line 368, in run
self.run_command("build")
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\dist.py", line 963, in run_command
super().run_command(command)
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
cmd_obj.run()
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\_distutils\command\build.py", line 131, in run
self.run_command(cmd_name)
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\dist.py", line 963, in run_command
super().run_command(command)
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
cmd_obj.run()
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\command\build_ext.py", line 88, in run
_build_ext.run(self)
File "C:\Users\proneon267\AppData\Local\Programs\Python\Python312\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 345, in run
self.build_extensions()
File "C:\Users\proneon267\AppData\Local\Temp\pip-install-tn4ge2dw\pillow_3e4dcb9162ff4e10a8de3cb36afc200c\setup.py", line 810, in build_extensions
raise RequiredDependencyException(f)
RequiredDependencyException: zlib
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "C:\Users\proneon267\AppData\Local\Temp\pip-install-tn4ge2dw\pillow_3e4dcb9162ff4e10a8de3cb36afc200c\setup.py", line 1015, in <module>
raise RequiredDependencyException(msg)
RequiredDependencyException:
The headers or library files could not be found for zlib,
a required dependency when compiling Pillow from source.
Please see the install instructions at:
https://pillow.readthedocs.io/en/latest/installation.html
C:\Users\proneon267\AppData\Local\Temp\pip-install-tn4ge2dw\pillow_3e4dcb9162ff4e10a8de3cb36afc200c\setup.py:45: RuntimeWarning: Pillow 9.2.0 does not support Python 3.12 and does not provide prebuilt Windows binaries. We do not recommend building from source on Windows.
lambda: warnings.warn(
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pillow
Running setup.py clean for pillow
Building wheel for toga-core (pyproject.toml): started
Building wheel for toga-core (pyproject.toml): finished with status 'done'
Created wheel for toga-core: filename=toga_core-0.1.dev7857+g2e16e16-py3-none-any.whl size=110419 sha256=87f4f0ed56b366bf6d149ab4682710075464f6058b3b1988c53a0d27812fc76c
Stored in directory: C:\Users\proneon267\AppData\Local\Temp\pip-ephem-wheel-cache-kjkgdpi_\wheels\a0\b3\10\261955d66d6fe65e067fcab312ab31a4edb6468c6442712453
Building wheel for toga-winforms (pyproject.toml): started
Building wheel for toga-winforms (pyproject.toml): finished with status 'done'
Created wheel for toga-winforms: filename=toga_winforms-0.1.dev7857+g2e16e16-py3-none-any.whl size=376805 sha256=f08dcead627dd84000e87aa2b8b94548b8978d292f9ed69e5f073718d437df7f
Stored in directory: C:\Users\proneon267\AppData\Local\Temp\pip-ephem-wheel-cache-kjkgdpi_\wheels\52\8f\0e\48cd06961cda5ec934e48d87bd5b7e8ff2e3e5200544e7c2f1
Successfully built toga-core toga-winforms
Failed to build pillow
ERROR: Could not build wheels for pillow, which is required to install pyproject.toml-based projects
Installing dev requirements...
Unable to install requirements. This may be because one of your
requirements is invalid, or because pip was unable to connect
to the PyPI server.
Log saved to E:\patch-20\testbed\logs\briefcase.2023_12_22-03_58_39.dev.log
Expected behavior
The test should run without failing to install the requirements.
Screenshots
No response
Environment
- Operating System: Windows 11
- Python version: 3.12.1
- Software versions:
- Briefcase:0.3.16
- Toga: latest fork of main
- ...
Logs
briefcase.2023_12_22-03_58_39.dev.log
Additional context
No response
Pillow version 9.2.0 doesn't have any Python 3.12 wheels on PyPI, so we should probably upgrade the testbed to a newer version.
Meanwhile, you can run the tests on Python 3.11 or older.
Since we do officially support Python 3.12, let's leave it open as a reminder to do the upgrade.
I'm not sure it is worth keeping this open.
The issue is that PIL 9.2 (the version pinned by testbed) doesn't have published Py3.12 wheels. However, if the host system was set up to compile wheels, it would work - the problem is that 99% of Windows users aren't set up to compile wheels. This is much leas prevalent as a problem on Linux and macOS. To my read, this isn't a bug in Toga; it's a known complication about wheel availability that is just how Python works (and breaks) in practice.
At some point, we'll upgrade the PIL version. I mean, we could bump to v10 now, if we wanted; the only issue would be wheel availability on Android, and that's an easy "just run the build script" fix. That will make this immediate bug go away; but it won't do anything to address the larger "windows boxes can't easily build wheels" issue.
Beyond doing a version upgrade right now, there's essentially nothing actionable here from the perspective of Toga; and I warrant we're not going to remember this ticket is open when we eventually update Pillow.
the problem is that 99% of Windows users aren't set up to compile wheels. This is much leas prevalent as a problem on Linux and macOS
It doesn't work on my macOS machine either, because there aren't any libjpeg headers available. A typical Linux machine will be similar, because most distributions put headers in a separate package to libraries.
RequiredDependencyException:
The headers or library files could not be found for jpeg,
a required dependency when compiling Pillow from source.
Please see the install instructions at:
https://pillow.readthedocs.io/en/latest/installation.html
So I think it's worth doing a Pillow update every time we add a new Python version. I've already built Android Pillow 10 wheels for Python 3.12, and I can do the other Python versions after I come back.
Ok - in which case, lets repurpose this ticket to track upgrading the Pillow dependency in testbed to 10.X; that gives us a clear success criteria.