fpm
fpm copied to clipboard
jordansissel
Support Wheel python binary format
See here for details about wheel.
Basically the feature should allow you to download a package from PyPi and turn it into a wheel.
I gotta second this. Wheel is rapidly becoming a standard in Python and is a great improvement over eggs. Wheel has been PEP'ed and it's official. Let's do this thing!
Why fpm does need to do this? There exist pip wheel tool that does exactly what is requested - "download a package from PyPi and turn it into a wheel".
Because fpm is the entry point of every artifact building process. Say you want to turn a package to a wheel and then package it as a deb/rpm with upstard/systemd configuration. It's easier to do with one tool. Especially if you use fpm-cookery.
I'm interested in implementing this, as I'm currently unable to repackage mxnet, mxnet-cu80, or tensorflow-gpu from their pip packages on pypi:
https://github.com/tensorflow/tensorflow/issues/6540
Based on the description here: https://pypi.python.org/pypi/wheel it seems that the .whl format is (in most cases) simply a zip file. Would an alright approach be unzipping this wheel (or installing it via pip install) and building any other packaging type as if the source was a zip?
Your proposal sounds like a good start!
On Fri, Oct 20, 2017 at 12:10 PM Mike Grimes [email protected] wrote:
I'm interested in implementing this, as I'm currently unable to repackage mxnet, mxnet-cu80, or tensorflow-gpu from their pip packages on pypi:
tensorflow/tensorflow#6540 https://github.com/tensorflow/tensorflow/issues/6540
Based on the description here: https://pypi.python.org/pypi/wheel it seems that the .whl format is (in most cases) simply a zip file. Would an alright approach be unzipping this wheel (or installing it via pip install) and building any other packaging type as if the source was a zip?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jordansissel/fpm/issues/587#issuecomment-338297337, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIC6tuZDDoQtdaU-XG3L-aT685wSr64ks5suPA5gaJpZM4BOS9W .
I've looked into this a little bit as well, I also want to repackage mxnet-cu80, which doesn't have a source package on pypi.
Would an alright approach be unzipping this wheel (or installing it via pip install) and building any other packaging type as if the source was a zip?
Almost. You also need to generate a new setup.py file, as that is not distributed with a wheel. To do this properly, you will need to parse the data contained in the .dist-info and .data directories inside the wheel. A source package had different files for that. Apart from this, there seems to be no difference between a source package and a wheel, as can be seen in PEP427.
To parse the metadata properly, you'll need to look into PEP426-2.0 and PEP426-3.0(The title is incorrect here!). The *.dist-info/metadata.json contains most information needed, but for some special functionality like installable scripts, you will need to look at the *.data directory as well, which is defined in PEP427, linked above.
I've written a python script that can generate a minimal setup.py file. That could be used as a starting point. If you want, I can discuss with my employer whether I can release it.
Maybe it would also be possible to take a look at the implementation of setuptools wheel building process and just reverse that?
Hi @iFreilicht , I'd be interested in this script. Trying to distribute as rpm's and tensorflow (for example) is only distributed as a wheel.
This may be of interest here: https://github.com/montag451/pypi-mirror
I was able to use the output of that script to get setup.py files in tarballs that I could then use FPM to generate packages with
I have run into a few PyPi dependencies that a python module I was attempting to package with fpm was unable to package as source packages where no longer being posted to PyPi, only wheels. I have used this tool as a workaround, it may be of interest. https://github.com/upciti/wheel2deb
I had to do some finesseing with sed on the version strings to make the packages match the package dependencies fpm creates, but it works.
