Modernize Python installation

[viktormalik]

At the moment, we always need to run pip install . when doing development. This comes with several drawbacks:

• the development process is complicated and confusing,
• installing Python packages in Nix development environment is complicated,
• Nix is dropping support for pip installer (https://github.com/NixOS/nixpkgs/commit/6c85fff302, https://github.com/NixOS/nixpkgs/commit/26ccdea3d7).

Especially the last item is a problem since it will prevent us from upgrading to newer releases of nixpkgs which contain e.g. LLVM 18.

I suggest we refactor our Python code to remove the above problems. An inspiration could be the Python's official modernization guide.

The necessary steps are roughly:

• replace setup.py by pyproject.toml,
• build CFFI manually instead of via pip install (preferably move it to CMake),
• move bin/diffkemp somewhere else (diffkemp/diffkemp?) so that all files doing import diffkemp.xxx are inside the diffkemp/ directory.

[DanielKriz]

I could be assigned to this issue as in my job I moved most of our python based project to pyproject.toml.

Regarding the CFFI. I think that we could consider using pybind11, it is a modern C++ way of handling interactions with python. It could be more powerful as it is supposed to work with native C++ and we wouldn't have to declare CFFI functions in extern "C".

Regarding the bin/diffkemp, I think we could get rid of it thanks to this feature in setuptools, thus we could simply have our diffkemp main function somewhere and let setuptools handle the creation of the executable. But I am not sure if this is supported inside nix. From a quick search I found dream2nix which from the examples seems capable of doing this.

[viktormalik]

Sounds good. Let's discuss on the meeting tomorrow.

[PLukas2018]

Solved by #344