Craigacp
Update the python setup.
** New versions of Python (3.10) were having issues witht the LICENSE reference. Changed to use the setup.cfg file instead which appears to have more flexibility.
** Removed the make commands from setup.py. It wasn't referencing the correct files.
As Python doesn't have namespacing in PyPI then using feast is going to collide with https://pypi.org/project/feast/ if they are installed together. So the package should probably be renamed to fstoolbox or something similar.
@Craigacp I was able to package the data in a whl. This can be uploaded to pypi. This is an individual package so when someone wants to use the python stuff they don't need to install the C libraries manually.
We can set this up with a github action to automatically push to pypi when you release.
Please let me know if you need help with that (but based on your current settings I may not be allowed to add actions).
Ok, that's good. I've not published anything on PyPI before, and I suspect that'll take a few attempts to get going.
The Makefiles for MIToolbox and FEAST currently dump the libraries in relatively unhelpful places, and I'm not sure if it would work on Windows & macOS, though presumably it's working on Linux where you are testing it. In my experience when distributing binaries people invariably want it in a platform which it isn't currently built for, this has happened for basically everything I've shipped binaries for at work and for open source things too. As a result I'll see how it behaves on Windows x86 & macOS (x86 and ARM64) before turning any of that on.
Sounds good. I am seeing some issues with trying to upload a linux wheel to pypi. This build will work locally on linux though. It appears some of the linked libraries are causing issues. I will try to figure that out in the meantime
@Craigacp I was able to successfully push this to pypi. You can find the page here: https://pypi.org/project/fstoolbox/. I haven't been able to get the CI to work with python3.6, 3.7 or 3.8 yet. The Image that CI is using must have too new of libraries. I think we can open another PR for something like that. Right now, CI will only work on my branch (for this PR). This is a super simple fix that we can make in the future. Let me know what you think about this! Also sorry for the mass commits. I tried to interactive rebase and its really messing up due to a change I pulled in from the master branch. We can squash some later if needed.
-- Edit -- Well it was working, but now the shared objects aren't properly installed (so it's not done yet).
@Craigacp Looks like I was able to get it to work. The pypi version 0.0.2 should work on a linux system. We can play with windows later.
How does this end up packaging the whl? It looks like there are a few extra so files created (in the Python 3.11 version which I pulled down from PyPI), in addition to libMIToolbox.so and libFSToolbox.so. There might also be a duplicate copy of libMIToolbox.so in fstoolbox.libs/libMIToolbox-423b447f.so.
I ran pip install fstoolbox in Python 3.10 in a Ubuntu 20.04 VM and it's unpacked the wheel over venv/lib/python3.10/site-packages in a way that means it can't find libFSToolbox.so.
>>> from fstoolbox import CMIM
failed to find libFSToolbox.so
I am still working on that. It looks like pypi isn't able to see the multiple whls even though I can see that they were pushed. I may have to try a manylinux1 whl which didnt work the first couple of times. I was able to see 3.10 work I will have to play with that one. I had to rewrite it a bit to find the so files since pypi wont allow me to install to /usr/lib. There are a couple of things we can try. Its close but not there apparently.
Given it's a fairly simple C project it might be easier to run the build of both C libraries with distutils which should understand how to package things. It may be doing that already given there are so files packaged in there, but the CDLL mechanism isn't finding them properly.