pdftopng icon indicating copy to clipboard operation
pdftopng copied to clipboard

About wheel_repair.py for Windows

Open myselfhimself opened this issue 3 years ago • 4 comments

Hello I have not tried your CLI wheel repair script yet, but it is fresher and slightly more advanced than machomangler which it wraps and leverages. Have you thought about seperating it into a pypi.org package, maybe with a more windows-related name ?

Related cibuildwheel Windows wheel repair issue: https://github.com/joerick/cibuildwheel/issues/459

myselfhimself avatar Nov 07 '20 11:11 myselfhimself

Have you thought about seperating it into a pypi.org package, maybe with a more windows-related name ?

I thought about it but didn't do it because the script has some limitations right now (which doesn't affect my very simple extension):

  • it assumes that there’s just one PYD file in the wheel which might not be the case for complex projects with multiple PYDs
  • it doesn’t look for DLLs in system directories, only the vcpkg installation directory (as I wasn’t sure what all directories to look into)

I would love to hear about your use-case before thinking about packaging this. Do you have multiple PYDs in the wheel you're trying to build, are you using vcpkg, and what are the build systems behind C/C++ code in your extension, etc. We can also get on a call if you're available.

vinayak-mehta avatar Nov 08 '20 09:11 vinayak-mehta

Hello, I think that those limitations are not big drawbacks and do not prevent from releasing your wheel repair project as a pypi.org package with its own Github/* repository, as long as you indeed mention those 2 known issues. Also publishing it in such a separate way would attract other people to come and make pull requests. Imperfection is sometimes perfect for now.

@zooba, you seem literate in this topic, would you have thoughts to share, if time allows it?

@vinayak-mehta My use case is for gmic-py, a Python binding around a C++ library named libgmic (or G'MIC) The project just generates a single .dll (or .so depending on the OS) which is a recompilation of libgmic (so, static in someway), with a light C Python API layer (in gmicpy.cpp), and depends dynamically on libpng, libjpeg, libtiff, zlib, libfftw, lipomb (OpenMP support), libcurl, not more shared libraries I think. Its only Python dependency is wurlitzer, which is a pure python package, allowing for automatic stdout/stderr redirections. The project has no init.py file or any pure-Python files for the end user, apart from pytests and setup.py.

So since I want to avoid pip install users to install all the dynamic libraries on Windows.. I am looking for a 1 command trick, hence .your wheel repair command. MSYS2 is the currently targeted environment, because our project leader liked compiling libgmic on MSYS2 which makes feel like on Linux on top of Windows... so I am continuing with MSYS2. Note that in an MSYS2 environment, .dll dependencies introspection can be done at least supercially with the unix-like builtin "ldd" command.

For now, this not yet cibuildwheel-enabled windows Github Action job log shows 2 artifacts which are :

  1. gmic-py's compiled .dll file for direction import gmic
  2. gmic-py's .whl file containing that same .dll file, but I fail to include other dlls for now at or before wheel-build-time, because I do understand which pieces of code and toggles to enable in setup.py / Manifest.in... If some tool would repair my build a wheel post-build time, without my need to fiddle with those manifest/setup.py package params, it would also be fine.

myselfhimself avatar Nov 09 '20 11:11 myselfhimself

Not sure what thoughts you want me to have...

I dislike this entire approach, regardless of operating system, and would rather people used stable/forward-compatible APIs. I don't want to have to debug the potential resource and name conflicts, and it bothers me that we take this approach to try and automatically fix issues that either need to be fixed by having conflicting packages communicate with each other, or by establishing package demands in a way that distributors and users can resolve it.

Those probably weren't what you were hoping for :) Might have to be more specific

zooba avatar Nov 12 '20 17:11 zooba

I must agree, if I understand well, that the machomangling step seems perfectionist to me... What I wished in a repair tool for Windows was just: 1) inspect the needed third-party .dll files, 2) put them in the wheel archive

myselfhimself avatar Nov 13 '20 11:11 myselfhimself