Upgrade numpy requirement

[aabmets]

Would it be possible to upgrade the numpy requirement of FEDOT to the latest version? My software requires the latest version of numpy and I'm unable to implement FEDOT into my software until FEDOT supports the latest numpy version.

[nicl-nno]

Hello, Mattias.

Which error do you have with latest (1.22.4?) numpy?

[aabmets]

@nicl-nno I'm using poetry package manager and poetry refuses to install FEDOT, because my software uses the latest pandas package, which requires the latest numpy package. I can't downgrade the numpy version, because then I would have to downgrade the pandas version. It is not possible to downgrade these packages, because doing so would reintroduce bugs into my software.

I'm either left with having to create a whole separate sub-project and venv for FEDOT and run that with subprocess.Popen and use interprocess communication, which is a very big inconvenience, or plead with FEDOT developers to bump the numpy version so that FEDOT could be used with latest pandas package.

I tried forking FEDOT with the intent of creating a patched wheel for myself, but I could not get the unittests to pass, because they were always throwing some errors, so I abandoned that option. Therefore I'm pleading with the FEDOT devs to bump the numpy version, before I undertake the huge task of creating a separate sub-project for FEDOT and having to implement inter-process communication.

EDIT: Maybe it's possible to configure the FEDOT numpy dependency in such a way that the numpy version to be installed depends on the Python version of the venv, if there is a concern with losing compatibility with legacy Python version <=3.9?

[nicl-nno]

Thank you for detailed description.

One more clarification: do you use the FEDOT 0.5.2 from pypi? In master version we have # Data numpy >= 1.19.* requirement and it works for unit tests in github actions.

And which exactly version of numpy do you installing?

[aabmets]

@nicl-nno Yeah, I was using FEDOT v0.5.2 from PyPI, because it's simpler to use with poetry, but I guess I could download the master branch and build that into a wheel to use as a file depencency. I'm using numpy = "^1.22.4" and pandas = "^1.4.2"

[nicl-nno]

OK, we will check if this combination of numpy and pandas works for us.

[aabmets]

@nicl-nno I found out that it's not possible to install FEDOT even with updated numpy and pandas requirements, because FEDOT depends on the DEAP package, which is incompatible with Python 3.10, which I am using, because their tensorflow requirements are obsolete, so even if you bump FEDOT numpy and pandas requirements, the library is still incompatible with Python 3.10 due to the DEAP package dependency.

[nicl-nno]

It looks like we never run FEDOT at 3.10 at all :-)

We will try to conduct such a experiment and provide some workaround. However, there is really a lot of problems for DEAP and Python 3.10 exists.

[aabmets]

@nicl-nno I'll see if I can contribute to the DEAP project to make it compatible with Python 3.10

[nicl-nno]

It will be great! At the same time, it looks like that DEAP project is quite abandoned (last PR merged in february).

[aabmets]

@nicl-nno Since I'm not familiar with the usage of DEAP within FEDOT, would it be a huge task to replace DEAP with PyGAD, for example? PyGAD is being actively maintained and also provides genetical evolutionary algorithms. So the question is, if you desire to maintain FEDOT, would you be willing to replace DEAP with another library, or see if DEAP can be updated to be compatible with Python 3.10?

[aabmets]

@nicl-nno I've created a new project for DEAP revival, you can find it at DEAP-er

[nicl-nno]

Great! I hope it will help you to run FEDOT with Python 3.10.

From our side, we will analyse it it posible to get rid of DEAP requirement at all. Unfortanately, it can not be replaced with PyGAD since we use SPEA2 selelection procedure for multi-objective tasks (and there is no such a procedure in PyGAD).

Feel free to ask any questions if needed.

[aabmets]

@nicl-nno Hey! I finished the beta version of DEAP port to Python 3.10. You can install it with: pip install deap-er

The interface of DEAP-er is the same as that of the old DEAP, so you just have to change the import statement from import deap to import deap_er and everything should just work.

Please let me know if it throws any errors.

[nicl-nno]

In the end, we removed DEAP and moved out optimizer to a different repo (see https://github.com/aimclub/GOLEM for details).

The support of Python 3.10 is implemented in both FEDOT/GOLEM.