pulp icon indicating copy to clipboard operation
pulp copied to clipboard

Please do not include binaries in the python package

Open yurivict opened this issue 5 years ago • 12 comments

https://pypi.org/project/PuLP/

Please depend on the project that builds those binaries.

Currently it just installs binaries that don't even match my system:

./work-py27/stage/usr/local/lib/python2.7/site-packages/pulp/solverdir/cbc
./work-py27/stage/usr/local/lib/python2.7/site-packages/pulp/solverdir/cbc/linux/32/cbc
./work-py27/stage/usr/local/lib/python2.7/site-packages/pulp/solverdir/cbc/linux/64/cbc
./work-py27/stage/usr/local/lib/python2.7/site-packages/pulp/solverdir/cbc/osx/64/cbc

My OS is FreeBSD, but I don't want binaries for it. It should use executables built from source.

yurivict avatar Sep 17 '18 20:09 yurivict

Yup I know where you are coming from but that functionality is mostly for windows users who don't have good package management systems.

Even for systems that do i don't know if there is a way to specify in a python package that it depends on cbc which is provided by some systems package (or maybe you have built it yourself)

I'm open to ideas for this but at the moment, but i feel that providing these binaries in the klugdy way that I do actually makes pulp a lot easier to install for 90% of our users.

stumitchell avatar Sep 17 '18 21:09 stumitchell

Maybe for system maintainers who want to build packages I should provide an alternative install which doesn't include these :-/

stumitchell avatar Sep 17 '18 21:09 stumitchell

On FreeBSD any package, including python-based ones, can depend on any other package. But I know, Windows doesn't have any packaging system to speak of. I wonder how do they even install things there.

yurivict avatar Sep 17 '18 21:09 yurivict

I'm far from being an expert, but conda was developed precisely to solve this problem. It looks like there is at least one conda recipe for Cbc (https://github.com/conda-forge/coincbc-feedstock) and there may be others. I've been in touch with a few people developing conda recipes for various COIN-OR packages. If I can find the bandwidth, this is something I'd like to try to develop. Perhaps only needed on Windows, but useful there.

tkralphs avatar Sep 17 '18 22:09 tkralphs

I guess I should also mention that there are also automatically built binaries for Windows here:

https://bintray.com/coin-or/download/Cbc

and a GUI installer as well (I haven't quite gotten to making this auto-updated, so it's a little out of date but works well):

https://www.coin-or.org/download/binary/OptimizationSuite/COIN-OR-1.8.0-win32-msvc12.exe

tkralphs avatar Sep 17 '18 22:09 tkralphs

In general, you should:

  1. Only install a binary on the architecture it is built for.
  2. Add the --with-system-cbc {full-path-to-the-cbc-executable} configure argument for all systems, and use the supplied executable. Also, do not install the binary if this argument is supplied.

yurivict avatar Sep 17 '18 22:09 yurivict

Honestly, I find it very convenient to have the binaries ready when I install pulp on a Windows machine.

fgenoese avatar Sep 18 '18 07:09 fgenoese

Yes, just use it on Windows, but there's no need to install windows binaries on linux or BSD.

yurivict avatar Sep 18 '18 07:09 yurivict

Binaries are nice but we really need them statically linked. Is there anyway to get them built automatically.

Stu

On Tue, 18 Sep 2018, 10:46 AM Ted Ralphs, [email protected] wrote:

I guess I should also mention that there are also automatically built binaries for Windows here:

https://bintray.com/coin-or/download/Cbc

and a GUI installer as well (I haven't quite gotten to making this auto-updated, so it's a little out of date but works well):

https://www.coin-or.org/download/binary/OptimizationSuite/COIN-OR-1.8.0-win32-msvc12.exe

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/coin-or/pulp/issues/180#issuecomment-422196264, or mute the thread https://github.com/notifications/unsubscribe-auth/AAtEWcL5gNjORBJAv4TfhDoD_vSSobUlks5ucCY1gaJpZM4Wsrh9 .

stumitchell avatar Sep 18 '18 20:09 stumitchell

@stumitchell The binaries I pointed to on bintray are automatically built and are statically linked. I just downloaded to a brand new Windows box and the Cbc binary runs with no installation. By the way, AMPL also distributes statically linked Windows binaries:

https://ampl.com/products/solvers/open-source/

and I know the JuMP/Julia guys are working on automated builds, too. So there's no shortage of good alternatives on Windows.

tkralphs avatar Sep 19 '18 01:09 tkralphs

Shouldn't https://pythonwheels.com/ fix this by allowing to create wheels of the cbc distribution for various OS?

https://realpython.com/python-wheels/

The wheels could be created and uploaded to PyPi automatically by github actions on release.

MLopez-Ibanez avatar Jul 29 '21 21:07 MLopez-Ibanez

For C extensions, wheels still don't seem to be a very good solution on Linux and OS X, but maybe they would work fine for packaging a statically linked binary. A conda package seems like a better solution overall for Linux and OS X (as well as for Windows, once we work out a few issues in the conda build on Windows for Cbc), as mentioned in #431.

tkralphs avatar Jul 30 '21 15:07 tkralphs