Autogenerate custom.mk?

[mikebentley15]

Perhaps it is possible to autogenerate the custom.mk file from the user's build system. This could perhaps be done by creating a wrapper around the used compiler(s) (and linker(s)) and capturing all files and flags needed to compile everything. This may be a very promising approach, as all that would be required is to replace the compiler used by the user's build system to use the wrapper script, which may be auto-generated by a flit command-line invocation (something like flit capture).

I think this would be quite the cool feature, and would reduce drastically the cost to entry. I don't know what kind of problems we could encounter in trying to implement this. I think the best way to get this at least mostly right is to find a few outside projects to use as test subjects. Get this working for a sufficiently simple case (such as MFEM -- that one is pretty easy to compiler), and then moving on to more complicated situations (such as Laghos or Lullesh).

[mikebentley15]

Here are some compiler capture tools I have found:

• CMake: (BSD license) with -DCMAKE_EXPORT_COMPILE_COMMANDS
• Build EAR or Bear: (GPL license) uses LD_PRELOAD and DYLD_INSERT_LIBRARIES to capture compilations "for clang tooling"
• scan-build: (LLVM license) can use build wrappers (for CC and CXX environment variables) and also with LD_PRELOAD and DYLD_INSERT_LIBRARIES
• compiledb: (GPL license) works for GNU Make, or also from a text file of echoed commands.

The CLion documentation describes how to create JSON compilation databases.

Perhaps instead of making a tool specifically for FLiT, we document how to create a JSON compilation database, and instead focus our efforts to being able to use such a database file. This would be a similar approach to the CLion documentation page. If we find that existing tools are not sufficient, or we need more information than what the compilation database provides, we may make an additional tool that can create a compilation database.

I'm envisioning something like

flit init --compilation-database=../../compile_commands.json

or for an existing FLiT directory,

flit update --compilation-database=../../compile_commands.json

which would generate custom-autogen.mk and a message to the console that the user can either manually merge or just replace custom.mk with this autogenerated file.

[mikebentley15]

Here is a good set of notes on compilation databases: https://sarcasm.github.io/notes/dev/compilation-database.html

[mikebentley15]

Another article on compilation databases: https://eli.thegreenplace.net/2014/05/21/compilation-databases-for-clang-based-tools

[mikebentley15]

I've started working on development here. I couldn't find a tool that does everything we need, but I found one that is close: scan-build. That code base is under the LLVM license which is very permissive.

I made a new tool called flit capture that uses scan-build as a dependency. You can find it in the issue189-flit-capture branch. The feature is not yet complete. It needs unit testing and documentation, but it seems to be functional based on my few manual tests. As part of this feature, I have added some fields to the JSON compilation database format specified by LLVM, specifically for use with FLiT in auto-generating custom.mk. Since it is a superset of what other tools depend on, it will probably work with tools that utilize a compilation database, or maybe not.