Minibook: Interfacing Fortran with Python

[awvwgk]

Tracking issue for creating a minibook on language intercompatibility of Fortran with Python.

This is a non-comprehensive list of method to interface Fortran and Python, feel free to edit this list:

• ctypes: dynamic loader for C libraries
• cffi: Python wrapper generator for C
• SWIG: Python wrapper generator for C
• numpy.f2py: wrapper generator for Fortran
• cython: compiler for Python C-extensions
• fwrap: wrapper generator for Fortran
• forpy: bindings to CPython
• f90wrap: wrapper generator for Fortran
• fmodpy: wrapper generator for Fortran
• F2x: template-based wrapper
• Forthon: wrapper generator for Fortran
• gfort2py

Aspects to cover:

• ease of use, constraints (bind(C) requirement, derived type support, finalization, ...)
• memory management, function callbacks
• build system integration (compiling of extension modules, distribution, ...)
• stability (frequent tool chain changes), portability (platforms, Python implementations), runtime overhead
• integration with OpenMP, MPI, CAF (mpi4py, ...)

[Beliavsky]

The Fortran Wiki has a Python page listing many of the packages above and few others.

Fortran should subsume other languages, not merely interoperate with them :).

But seriously, one way to speed up Python with Fortran is to translate it to Fortran with Pyccel, an active project.

[ivan-pi]

I've added a few from the Python wiki. It would be interesting to learn what the standards committee thinks of this. There seems to be huge interest in using Fortran from Python (I suppose either for reasons of calling legacy code, or speeding up critical parts).

Personally, I've only ever used the Cython route. On Linux it worked very well, but on Windows I struggled with binary compatibility issues. It's not fully clear to me how the people who support numpy and SciPy manage to pull it off. I know I've read some interesting blog posts about it, but couldn't find them anymore.

[ivan-pi]

Another route is to use pybind11 and follow the path Fortran --> C --> C++ --> Python. It was demonstrated by @hsnyder on Discourse: https://fortran-lang.discourse.group/t/iso-c-binding-pass-an-array-from-c-to-fortran-edit-python-interop-content/514/8

[kjelljorner]

Following from the discussion here, I would be interested in contributing to this. As I noted in the last post there, I think it would be useful to work on this in a real-time collaborative markdown environment (e.g. GitBook) at first and then transfer to fork/pull request route once its more finalised.

I see different levels of complexity and different use cases which could be covered here. For something on Fortran-lang.org I argue that focusing on established and well-maintained tools is the way to go. There are many packages out there that are either abandoned or have rapidly changing APIs. As I see it, there are two practical approaches to accessing Fortran code from Python:

• Explicitly write explicit C interface, build shared library and access via ctypes, cffi, Cython etc.
• Use interface generators which take care of everything under the hood and directly produces importable Python objects. Here the most mature would be f2py which is used in production-grade code like numpy and scipy.

Code generators like Pyccel, Numba, Cython are in my view a bit of a different topic as their main goal is to enable users to write Python-like code that runs faster and not to actually generate Fortran code that could be used for something else. The user would be fairly agnostic whether it is Fortran, C or something else that is generated under the hood.

[hsnyder]

I’m actively working on mixed Python-Fortran projects with work right now.. I’ll keep this thread in mind and share anything I produce that I think might be useful. At the moment what I’m doing involves fortran calling python, rather than the opposite way around. It will be a few days before it’s useable outside my immediate application though.

[kjelljorner]

I made some prototype page on HackMD. If this way of working looks attractive to people I could create a "team" and we could create the minibook there.

[awvwgk]

Sure, I would be happy to contribute an example with intermediate complexity involving (polymorphic) derived types.

[LKedward]

Thanks @kjelljorner! That looks like a nice way of working on the minibook

[kjelljorner]

I created a space on HackMD to work on this. It can be accessed only if you login, while the individual notes themselves can be shared to anyone. @awvwgk and others, If you create an account and let me know the username I can add you to the team as admins. There's a quite nice comments function that we can use to discuss different parts of the text.

[awvwgk]

If you create an account and let me know the username I can add you to the team as admins.

Thanks, I'm going under @awvwgk there as well.

[kjelljorner]

I'm currently working on a Python interface to a legacy Fortran code, involving sending Python callback functions to Fortran. Also considering the options for multi-platform packaging on PyPi using scikit-build + cmake and cibuildwheel. Maybe would be a good time to try to revive this effort @awvwgk?

[awvwgk]

I currently have shifted my priorities from the Fortran-lang webpage to the fpm documentation (see https://awvwgk.github.io/fpm-docs). Hopefully, the framework explored there will find adoption for other Fortran-lang resources in the future (stdlib? minibooks?), especially since we now actually have first class internationalization support there.

[kjelljorner]

I've seen that - it looks really good! I might add some of the work on callbacks and error handling that I have been investigating over the next few weeks and then see if there is anyone on the Discourse that wants to join in.