gmic-py
gmic-py copied to clipboard
myselfhimself
cpython: add support for windows
- [x] add msys2 to Github Workflow recipe
- [x] install all libraries
- [x] install all python dependencies (pkg-config, wurlitzer)
- [x] complement setup.py with msys2/mingw libomp libs and include paths
- [x] build .dll sucessfully (debug)
- [ ] build .dll sucessfully (optimized)
- [ ] have test suite running green
- [ ] have someone test GDI display
- [x] build .whl successfully (without missing .dll repair in place)
- [ ] build .whl successfully (.dll repair in place)
- [ ] build .whl successfully (optimized, .dll repair in place)
- [ ] make set up for python 3.6-3.9 or so (cibuildwheel?)
- [ ] release for python 3.6
- [ ] release for python 3.7
- [ ] release for python 3.8
- [ ] release for python 3.9 (?)
- [ ] announce release
- [ ] work back in master branch so that tag-based releases trigger both windows and linux etc.. releases
This is deprioritized in favor of making first a fully fledged gmic-py v1 for Linux (and Mac OS)
Since we want to bring G'MIC into Blender thanks to this gmic-py binding, looking at Blender's Open Data shows that 2/3 of Blender users are coming from Windows... So Windows support will be needed pretty soon :)
Per: https://opendata.blender.org/
Consider using cibuildwheel with github action support. See related issue having good progress: https://github.com/joerick/cibuildwheel/issues/159
Attempting to build for Win64 platform. Major errors, not sure how to get going. Thanks for the efforts to get GMIC Python to work in Blender. I will build for GraphicAll.org when it works.
Hello @paulhart2 Thank you very much for the errors reporting. I see how to correct them but will come back to you once I have tried compilation on the Github Actions CI platform which probably has MSVC like you, I will ping you back if I struggle or succeed. Note that gmic-py is indeed intended for any Python 3.5+ environment, but if imported as is within Blender3d, developers using it will need higher skills in order to grab Blender buffers and pipe them into G'MIC or will have to work files only. The latest binary builds of gmic-py are on pypi.org here and I will care that this pypi repository always remains up to the latest version of the underlying G'MIC library as long as I am contracted on this project.
What you may want for Blender3d is maybe the G'MIC Blender plugin, which is to expose more than 500 compositing or texture nodes, but for now that plugin has only Linux support and does not even create 1 node.
I wanted to finish a work on https://github.com/douglaskastle/blender-addon-tester first (very simple MacOSX support). This is a matter of 1/2 day I hope.
Could you tell me what is GraphicAll.org for? Do you build just Blender 3d or a variety of open-source graphics software? How would you see g'mic as a python binding or Blender plugin in it? Thanks
WIP Windows build job on Github Action: https://github.com/myselfhimself/gmic-py/runs/526438299?check_suite_focus=true
- [ ] setup.py needs removal of pkgconfig checks for Windows
- [ ] wget is not an available command, maybe use it non-aliased WindowsShell form
- [ ] ensure GDI is used in place of X11
@paulhart2 is MSVC compilation a must for you? gmic-py would compile best on Windows using MSYS2 instead from what I have heard. Though I do not want to prevent so many Windows users from self-compiling gmic-py (I intend to release just source and as to Windows, gmic-py Windows binary pre-compiled packages)
Based on this 2017 post on compiling G'MIC on Windows using either MSYS2+Mingw or MSVC, given that the latter MSVC option would probably force to disable OpenMP, I prefer not to even try using MSVC compilation... So Windows Python developers wanting to compile their own binding will have to follow compiling instructions or get inspiration from my upcoming compilation scripts.
I will let you know once I manage to compile gmic-py for Windows, hopefully this week.
Python for Windows is compiled with a specific MSVC version (see Python wiki). The same MSVC version must be used to compile extensions. I'm not certain whether this applies to dynamically loaded libraries that are used by the extension.
Thank you @jfolz , MingW is also possible up to Python 3.4 according to that same Python wiki.... We could hope that it actually does work for Python versions not just equal to but greater than 3.4.
Possible MSVC versions that I could use (if OpenMP eventually would want to work on it) on Github Actions (I do not to do the windows/packaging locally at all) are listed here for Windows Server 2016, 2019.
This will really be my first compiling a complex project for Windows in the "blind" of a remote CI toolchain. Keep suggesting ideas or try PRs (Github Actions do run on PRs as well) etc..
Some alternative compilers exist like MinGW, but incompatibilities may occur with a CPython official distribution that is built with Microsoft Visual C++.
Essentially: there be dragons. Segfaults that suddenly occur in 3rd-party libraries, and thus almost impossible to debug.
It has been many years already that gmic compiles and runs well with MSYS2/MINGW, Gimp for Windows also follows that path... Let us experiment, I have a pytest suite that runs for all OSes Windows users such as paulhart2 will be able to test for sure.
Out of curiosity here is a github action to set up msys2 Most important for me is that final final library or wheel needs no extra depencencies apart from what's bundled.
Windows tool similar to macos delocate and linux auditwheel repair, for including DLLs: https://discuss.python.org/t/need-auditwheel-like-utility-for-other-platforms/2028/6
Related dependencies checker for windows: https://lucasg.github.io/2018/04/29/Dependencies-command-line/
Focusing on this for today on until its finished. I will ensure that the virtual machine for building and that for testing are different, namely so that the testing environment does not have any .dll to cheat.
https://github.com/d0m3nik/pycimg/blob/master/.travis.yml shows a windows build using CI Build wheel elegantly, for a similar project (CImg is the backbone of GMic)
@paulhart2 welcome aboard for attempting to compile gmic-py on Windows / MSYS2 by hand firsthand I can then help transpose our efforts into config-as-a-code Github Workflow files containing build and release jobs for every new gmic-py version
Good news, MSYS2 / pacman is already on the Windows images for Github Actions: https://github.com/actions/virtual-environments/blob/main/images/win/Windows2019-Readme.md
Related to https://github.com/msys2/MINGW-packages/issues/7233 OpenMP lib without pkg-config .pc "manifest"
Ideas for a solution to bundle DLLs: https://vinayak.io/2020/10/23/day-53-a-tool-to-list-pe-file-dependencies/ https://stackoverflow.com/questions/35112511/pip-setup-py-bdist-wheel-no-longer-builds-forced-non-pure-wheels/36886459#36886459