Pillow
Pillow copied to clipboard
Windows build improvements
For #6369, close #6554, close #553
In an effort to sync the dependencies available on Linux/macOS (python-pillow/pillow-wheels) and Windows (test-windows.yml) ahead of the next release (#6460), I've compiled a list of dependencies (including indirect ones) below, and included some updates for Windows in this PR.
The full list is: (click to expand)
-
libjpeg
via its forklibjpeg-turbo
version 2.1.4 is included on all platforms -
zlib
version 1.2.12, included on all platforms -
libtiff
version 4.4.0, included on all platforms- uses
zlib
andlibjpeg
on all platforms - Linux/macOS uses
lzma2
version 5.2.6 fromxz
; added on Windows in this PR and added test - no platform uses
webp
because that is built afterlibtiff
on both platforms; swapped on Windows in this PR and added test - no current wheels include
deflate
,jbig
,libjpeg12
,lerc
,zstd
, but cgohlke mentioned at least LERC being included in his wheels in https://github.com/python-pillow/Pillow/issues/5573#issuecomment-872691741
- uses
-
webp
version 1.2.4, included on all platforms- AFAICT its dependencies
libjpeg
,libpng
,libtiff
, andgiflib
are only used for binary conversion utilities, notlibwebp
. However, on Linux/macOS multibuild ensures these are built beforewebp
. For Windows I movedwebp
to be built beforelibtiff
.
- AFAICT its dependencies
-
freetype
version 2.12.1, included on all platforms- uses
zlib
andlibpng
on all platforms - uses
harfbuzz
only on Windows because the cyclic dependency is tricky to resolve with GCC - only Linux/macOS uses
bzip2
version 1.0.8. I tried adding it on Windows, but the driverftbzip2
appears to have been left out of the Visual Studio project files. It is only used for.pcf.bz2
fonts which aren't that common anyway (especially on Windows). -
brotli
version 1.0.9 added in https://github.com/python-pillow/pillow-wheels/pull/320 for Linux/macOS and this PR for Windows
- uses
-
lcms2
version 2.13.1- AFAICT its dependencies
libjpeg
,zlib
,libtiff
are only used in binary utilities, not the library
- AFAICT its dependencies
-
openjpeg
version 2.5.0- according to the readme and build files its dependencies
zlib
,libpng
,libtiff
,lcms2
are only used in binary utilities, not the library; changed the build flag used on Windows to reflect this (makes builds marginally faster).
- according to the readme and build files its dependencies
-
harfbuzz
version 5.1.0 included on all platforms with its dependency FreeType -
xcb
is included on Linux/macOS; not very useful on Windows -
fribidi
is linked at runtime, support shim is enabled on all platforms -
libimagequant
is only used in tests, not included in wheels
Changes proposed in this pull request:
- Add
xz
, used bylibtiff
- Build
webp
beforelibtiff
rather than after so thatlibtiff
can use it - Add tests for reading TIFF files with LZMA and WEBP compression, generated with ImageMagick
- Add
brotli
forfreetype
with a WOFF2 font test for #6554 / https://github.com/python-pillow/pillow-wheels/pull/320 - Append license files to
LICENSE
before building wheels (on GHA only)- I added all included licenses when a readme provided multiple choices, which I later realised was a rejected suggestion for the Linux/macOS wheels. This is both the FTL and GPLv2 license for FreeType, as well as the BSD license mentioned in libjpeg-turbo. Let me know if I should remove these.
- Restore the
py_vcruntime_redist
variable. Although I previously thought it was no longer necessary, it turns out this is only true for recent builds of Python (such as the ones on GHA). It is still necessary for older builds such as the one on AppVeyor (I noticed it was emitting a linker warning). It can be safely removed again when Python 3.9.0 is the oldest supported release: https://github.com/python/cpython/blob/v3.9.0/Lib/distutils/_msvccompiler.py#L243
This is enough to get the Windows and Linux/macOS mostly in sync, with bzip2
for freetype
, and xcb
not enabled on Windows, and webp
for libtiff
not enabled on Linux/macOS.
Considerations for future PRs:
- cgohlke's wheels included more
libtiff
dependencies (e.g. LERC), might be worth adding on all platforms. - ~~Add
brotli
for #6554~~ - Due to static linking on Windows, combining
_webp
and_imaging
could save about 0.5MB per wheel with thelibtiff
changes above. - Additional testing (e.g. https://github.com/python-pillow/Pillow/issues/5573#issuecomment-877752592); I'm working on a separate PR
-
libpng
is failing to detect SSE2 support, should be safe to patch in (even Windows 7 requires SSE2 now) -
harfbuzz
build warning:HarfBuzz has a Meson port and tries to migrate all the other build systems to it, please consider using it as we might remove our cmake port soon.
-
openjpeg
suggests enabling SSE4.1/AVX2 if available, probably not safe to enable yet
no current wheels include deflate, jbig, libjpeg12, lerc, zstd
cgohlke's wheels included more libtiff dependencies
libtiff
included in the Windows wheels was built with libjpeg-turbo
, libwebp
, libdeflate
, zstd
, lerc
, and lzma
. JBIG-KIT
is GPL licensed.