Reduce inodes count

[hiker]

In our CI pipeline we are constantly running out of inodes (if a CI run fails, there is no cleanup). I expect that Fab will put additional pressure on the number of inodes, since for each (Fortran) input file, it will create:

1. a preprocessed file (even if the input file is .f90, it will create a copy so that the whole build is done from build_dir only)
2. an .an file
3. two .mod files (one with hash, one without)
4. a .o file

Though it also does save by only compiling the absolute minimum number of files. A rough analysis of one of my gungho builds:

930 source files
4676 build files

1156 *90 files (note that PSyclone will create two Fortran files for one X90 input file)
1675 *mod files (that's two per *90 file - hashed and non-hased version. Not all *90 files will probably be used in gungho, so that's why we have only around 837.5 files ... and no idea about why this is an odd number, best guess would be left-over mod files, since I rebuild in the same directory, and my fab development version sometimes crashes before cleanup??)
820 *o files
1023 .an files (analysis files)

So, of the 5606 overall files, the .hashed mod file and the .an file might not need to be written, saving around 1840 inodes, or around 33% (Fab uses internally cached information to get the analysis information from the analysis step to the compilation step, so writing the files is not really required in a successful build).

Of course, if the files are not written, and a run crashes, important information might be missing.

I am just raising this as a question (with no intention of working on it in the near future :) ): is a reduction of inodes count important for other sites?

There are also different (though more complex to implement) ways of reducing this, e.g. all .an files could be stores in a single file (sorting out concurrent writes from sub processes etc etc).

[t00sa]

We've seen this type of problem with fcm on some of our older platforms and, related, had problems where debug versions in built temporary locations have been unable to load symbol information. How about these options as potential solutions?

• An option to remove the build fields at the end of a run
  • If you are not going to running an incremental build, e.g. in a CI environment, the files don't have much value
  • If you are building a production run, the object files aren't needed once the build has completed
• An option to create a tar file of the object files at the end of a build
  • The tar file could be unpacked to support new incremental builds
  • The tar file could also be unpacked to support debugger listings

[hiker]

I like both ideas - as in, we should consider supporting both: an optional tar-ball, and an optional cleanup of all files (not only the files not used in the current build, which is what Fab does atm).