ford
ford copied to clipboard
Fortran-FOSS-Programmers
Documentation generation on my local machine and on github gives different outcomes
Hi Peter,
Thank you again for solving Issue #638. With Ford 7.0.6, I can generate the documentation without any problem on my local Mac. However, when my co-worker attempts to do it in github (same code), there are error messages.
https://github.com/GEOS-ESM/MAPL/actions/runs/8601316694/job/23568290998
Do you know if there is any setting we need to have to make it work?
Thanks.
Jules
In case you cannot access the above page, I pasted below part of the Github output:
Preprocessing /home/runner/work/MAPL/MAPL/pfio/tests/Test_Attribute.pf
Parsing files ━━━━━ 100% 693/… 0:00… 0:0… pfio/tests/Test_Attribute.pf
Correlating information from different parts of your project...
Warning: Could not identify ancestor module ('mapl3g_GeomManager') of submodule
'GeomManager_smod' (in 'GeomManager_smod.F90').
This is usually because Ford hasn't found 'mapl3g_GeomManager' in any
of the source directories.
Warning: Could not identify ancestor module ('mapl3g_MaplGeom') of submodule
'MaplGeom_smod' (in 'MaplGeom_smod.F90').
This is usually because Ford hasn't found 'mapl3g_MaplGeom' in any of
the source directories.
Warning: Could not find base type ('MpiTestParameter') of derived type
'ESMF_TestParameter' (in 'ESMF_TestParameter.F90').
If 'MpiTestParameter' is defined in an external module, you may be able
to tell Ford about its documentation
with the `extra_mods` setting
Warning: Could not find base type ('AbstractExportExtension') of derived type
'RegridExtension' (in 'RegridExtension.F90').
If 'AbstractExportExtension' is defined in an external module, you may
be able to tell Ford about its documentation
with the `extra_mods` setting
Warning: Could not find base type ('MpiTestCase') of derived type
'ESMF_TestCase' (in 'ESMF_TestCase.F90').
If 'MpiTestCase' is defined in an external module, you may be able to
tell Ford about its documentation
with the `extra_mods` setting
Warning: Could not find base type ('ApplicationMode') of derived type
'ModelMode' (in 'ModelMode.F90').
If 'ApplicationMode' is defined in an external module, you may be able
to tell Ford about its documentation
with the `extra_mods` setting
Warning: Could not find base type ('Regridder') of derived type 'NullRegridder'
(in 'NullRegridder.F90').
If 'Regridder' is defined in an external module, you may be able to
tell Ford about its documentation
with the `extra_mods` setting
Warning: Could not find base type ('ApplicationMode') of derived type
'ServerMode' (in 'ServerMode.F90').
If 'ApplicationMode' is defined in an external module, you may be able
to tell Ford about its documentation
with the `extra_mods` setting
Warning: Could not find base type ('StringVector') of derived type 'DirPath' (in
'MAPL_DirPath.F90').
If 'StringVector' is defined in an external module, you may be able to
tell Ford about its documentation
with the `extra_mods` setting
...done in 7.093s
Processing comments ━━━━━━━━━━━━━━ 100% 113670/113670 0:00:00 0:01:15 buffer
Creating HTML documentation... done in 0.543s
Generating graphs ━━━━━━━━━━━━━━ 100% 19532/19532 0:00:00 0:11:06 zero_pad
Error rendering page '/home/runner/work/MAPL/MAPL/docs/Ford/mapl3-doc/type/couplermeta.html'
Traceback (most recent call last):
⠙ Creating search index 1% 191… 0:23:43 0:00:… type/couplermeta.html
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/ford/output.py", line 351, in html
return self.render(self.data, self.proj, self.obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/ford/output.py", line 481, in render
return self.template.render(
^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/jinja2/environment.py", line 1301, in render
self.environment.handle_exception()
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/jinja2/environment.py", line 936, in handle_exception
raise rewrite_traceback_stack(source=source)
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/ford/templates/type_page.html", line 3, in top-level template code
{% extends "base.html" %}
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/ford/templates/base.html", line 128, in top-level template code
{% block body %}
^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/ford/templates/type_page.html", line 96, in block 'body'
{{ macros.bound_info(bp) }}
^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/jinja2/runtime.py", line 777, in _invoke
rv = self._func(*arguments)
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/ford/templates/macros.html", line 380, in template
{{ binding_summary(bind) }}
^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/jinja2/runtime.py", line 777, in _invoke
rv = self._func(*arguments)
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/ford/templates/macros.html", line 298, in template
{{ proc_line(proc, proto=proto) }}
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/jinja2/runtime.py", line 777, in _invoke
rv = self._func(*arguments)
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/ford/templates/macros.html", line 463, in template
{{ deprecated(proc) }}
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/jinja2/runtime.py", line 777, in _invoke
rv = self._func(*arguments)
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/ford/templates/macros.html", line 170, in template
{%- if entity | meta('deprecated') -%}
^^^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: Unknown entity 'finalize': This likely means an error in parsing, please check that this file compiles with a Fortran compiler
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.2/x64/bin/ford", line 8, in <module>
sys.exit(run())
^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/ford/__init__.py", line 489, in run
main(proj_data, proj_docs)
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/ford/__init__.py", line 473, in main
docs = ford.output.Documentation(proj_data, proj_docs, project, page_tree)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/ford/output.py", line 248, in __init__
self.tipue.create_node(page.html, page.loc, page.meta)
^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages/ford/output.py", line 353, in html
raise RuntimeError(
RuntimeError: Error rendering 'couplermeta.html':
File "GenericCouplerComponent.F90": type "CouplerMeta"
Unknown entity 'finalize': This likely means an error in parsing, please check that this file compiles with a Fortran compiler
Error: Process completed with exit code 1.
Hi Jules, my initial guess is that this is something to do with checking out the code -- Ford can't find some files that are maybe in an external library or git submodule perhaps?
If you try a clean checkout locally, do you see the same issue?
@ZedThree Here is the odd thing. If you look at the GitHub Actions output you'll see (for example):
Preprocessing /home/runner/work/MAPL/MAPL/generic3g/couplers/esmf-way/CouplerMetaComponent.F90
Warning: error when preprocessing
/home/runner/work/MAPL/MAPL/generic3g/couplers/esmf-way/CouplerMetaComponent.F90
:
cpp -traditional-cpp -E -DUSE_MPI=1 -DBUILD_WITH_PFLOGGER=1
-DBUILD_WITH_EXTDATA2G=1 -DUSE_FLAP=1 -DH5_HAVE_PARALLEL=1 -DTWO_SIDED_COMM=1
-DMAPL_MODE=1 -I/home/runner/work/MAPL/MAPL/include
-I/home/runner/work/MAPL/MAPL/gFTL/install/GFTL-1.13/include/v1
-I/home/runner/work/MAPL/MAPL/gFTL/install/GFTL-1.13/include/v2
/home/runner/work/MAPL/MAPL/generic3g/couplers/esmf-way/CouplerMetaComponent.F90
/home/runner/work/MAPL/MAPL/generic3g/couplers/esmf-way/CouplerMetaComponent.F90
:2:2: fatal error: Generic.h: No such file or directory
2 |
| ^
compilation terminated.
Reverting to unpreprocessed file
However, we have this file excluded, we think:
exclude: **/EsmfRegridder.F90
...
**/generic3g/couplers/esmf-way/CouplerMetaComponent.F90
Note that we also tried:
../../generic3g/couplers/esmf-way/CouplerMetaComponent.F90
and it still didn't exclude it.
Is there perhaps something we are doing wrong to exclude that file? Maybe if we get the syntax right, we can figure out all the other issues...
This is actually a bug, the paths in the settings file are supposed to be relative to the setting file, but (at least) the exclude paths get processed relative to where you run ford, and for some reason **/filename doesn't match filename.
For now, a workaround is to just run ford in the same directory as the settings file, that is, run cd docs/Ford; ford mapl3docs-with-remote-esmf.public_private_protected.md. This seems to work for me
Some notes for myself:
This is here: https://github.com/Fortran-FOSS-Programmers/ford/blob/8e05e458c4691a4d2c563454fbf75da3248de864/ford/fortran_project.py#L124-L127
When ford is run at the top level, then:
fnmatch(os.path.relpath(src), exclude)
becomes:
fnmatch('generic3g/couplers/esmf-way/GenericCoupler.F90', '**/generic3g/couplers/esmf-way/GenericCoupler.F90')
which is False -- which feels a little surprising to me! It might be better to switch to glob.glob(exclude, recursive=True) for pruning the file list, but also os.path.relpath needs to have the second argument with the path of the settings file here
This is actually a bug, the paths in the settings file are supposed to be relative to the setting file, but (at least) the exclude paths get processed relative to where you run
ford, and for some reason**/filenamedoesn't matchfilename.For now, a workaround is to just run
fordin the same directory as the settings file, that is, runcd docs/Ford; ford mapl3docs-with-remote-esmf.public_private_protected.md. This seems to work for me
Ah ha! And I bet both @JulesKouatchou and myself always run ford from the docs/Ford directory because of course we would. But my Github Action didn't do this because, well, I got something working and went hands off.
I'll test this soon.
--
A query @ZedThree: assuming this all works, what is the "right" way to exclude a specific filename in one subdir but not another? Is it:
**/dir1/dir2/file
or:
../../dir1/dir2/file
(since we know exactly where things are relative to the ford control file)
Note we probably have to be specific like this with subdirs, because MAPL3 is slowly becoming based more on path so we might have:
generic3g/foo/bar/module.F90
generic3g/hah/lol/module.F90
and maybe hah/lol is currently a stub that ford can't process (because it's not valid Fortran), but foo/bar is real code that it can. (We do not build hah/lol/module.F90 because it's not in our CMakeLists.txt but ford, of course, is not processing the CMake, it's just looking for Fortran!)
A query @ZedThree: assuming this all works, what is the "right" way to exclude a specific filename in one subdir but not another? Is it:
The benefit of the ** notation is that you can exclude similar directories no matter where they appear in your project structure.
The second one is perhaps a bit more explicit, with the downside of maybe being more fragile if you ever rearrange your project directories.
@ZedThree Makes sense. We'll stick with the **.
And good news! With your workaround--and some more excludes--we have some docs: https://geos-esm.github.io/MAPL/mapl3-doc/
🎉
Now it's time to start enforcing style, etc. as bits are leaking into places. 😄
MAPL is a foundation layer of the GEOS architecture, whose original purpose is to supplement the Earth System Modeling Framework (ESMF)