GCC: Support .d files

[bklarson]

Some build systems use -M arguments to GCC to generate a .d file, which specifies file dependencies.

Currently buildCache will generate these .d files in the .buildcache/tmp/ folder when running the preprocessor. We need to cache these files alongside the object files and repopulate them when retrieving object files from the cache?

Would it be easiest/best(/possible) to run each preprocesser job into a separate /tmp/ folder, and save off all extra-generated files? This would let us not worry about parsing the -M arguments to gcc. Or is there a better approach?

[mbitsnbites]

I think that the correct solution would be to:

1. Don't generate the .d files as part of the preprocessor run (that's currently an oversight in the wrapper) - only as part of the full run (upon a cache miss).
2. Add the .d files to the cache, and reproduce them upon a cache hit.

That is, if there are no problems with absolute paths or similar. When I did a test run with -MMD -MP (without BuildCache) I got .d-files next to my .o-files, containing only relative paths, which should work fine with a cache.

[asheplyakov]

That is, if there are no problems with absolute paths or similar.

Actually there are two path related problems.

1. Absolute paths to sources and "system" headers in .d files
2. Location of .d files themselves

When I did a test run with -MMD -MP (without BuildCache) I got .d-files next to my .o-files

GNU autotools put .d files into a different directory. Depending on autotools version and project configuration (subdir-objects) this might be either $top_builddir/.deps or $builddir/.deps

So it's very easy to mess up the dependencies.

On the other hand caching .d files won't really improve the performance. buildcache needs to run the preprocessor anyway, and this reads every #include file. Recording all #includeed files and printing them out as a makefile fragment doesn't require lots of CPU/IO.

buildCache will generate these .d files in the .buildcache/tmp/ folder when running the preprocessor

Perhaps buildcache should run the preprocessor in the build directory instead?

[mbitsnbites]

Note to self: The .d files probably need special handling in the new direct mode, since we do not run the preprocessor.

[mikehardy]

Following here - in my testing for my buildcache-action against the type of builds Xcode runs on react-native projects (at least, those were just what I tested) - the lack of .d files were I think the root cause of failures on second compile (warm cache) with direct mode enabled. With direct mode not enabled, builds worked