ford

ford copied to clipboard

Was wondering if there were any possibilities for using multiple cores to speed up the overall documentation generation (e.g., in the reading/parsing, or the html generation, or the calls to graphviz, etc.) This would be quite useful for very large projects.

I should be able to parallelize parts of the process (although definitely not all of it, as there are certain stages where things have to be processed in a specific order). However, I'm not familiar with Python's multicore abilities, so it would take some reading up.

@cmacmackin

Chris, I am using multiprocessing in FoBiS to speedup the compilation of each source file concurrently: I just create a pool of (co)workers. It is a dirty (old for Python) approach, but it scales quite good. You should be able to find more modern and Pythonish solutions. If you need help on this I will happy to try to study a solution with you, it could be a nice occasion to contribute to FORD!

P.S. I am studying FIAT in these days...

I've had a quick stab at adding some multiprocessing based parallelisation to ford in a branch of my fork. Currently this is mostly disabled as it doesn't quite work as intended in all places (the only thing successfully parallelised is the output of graphs).

This has been partially implemented in some pull requests, but doubtless more could be done.