py-fortranformat icon indicating copy to clipboard operation
py-fortranformat copied to clipboard

Reading performance

Open brendanarnold opened this issue 4 years ago • 2 comments

This looks like a great project that I would very readily use in my own codes (I work with a lot of Fortran codes and Fortran outputs).

The one thing that stops me from using the FortranRecordReader/Writer classes here in most of my own projects is that they're simply too slow for the size files that I'm trying to parse. As an example, in one of my projects I have a very simplistic parser that supports a tiny subset of permissible formats (but currently covers all of my use cases for that project). [https://github.com/swails/ParmEd/blob/master/chemistry/amber/amberformat.py#L22-L168 if you're interested]

Replacing my primitive code with the classes here results in a 5x-6x slowdown (file reading goes from 4 seconds to 17 seconds on a medium-sized example file). I understand the challenges in making codes both flexible and fast. I filed this issue in case you wanted to hear feedback like this. Feel free to close this issue as wontfix or invalid if either time constraints or project scope preclude working on performance enhancements.

brendanarnold avatar Mar 14 '21 17:03 brendanarnold

The way they are implemented is a port of the GFortran code which is really complex and 'bit pushy' C-code. I could make this a lot simpler once I have a good test suite in place.

brendanarnold avatar Apr 02 '21 17:04 brendanarnold

Works started at https://github.com/brendanarnold/py-fortranformat/tree/issues/9-performance

brendanarnold avatar Feb 21 '22 22:02 brendanarnold