llvm icon indicating copy to clipboard operation
llvm copied to clipboard

[SYCL] ccache is invalidated by random file names generated for offloaded code bundles

Open psalz opened this issue 3 years ago • 7 comments

Describe the bug

In its default "direct mode", ccache examines the preprocessor output to determine all dependencies for a given translation unit. In doing so, ccache gets tripped up by the randomly generated filenames for offloaded code bundles. The preprocessor output for a small example program will look something like this (excerpt):

// __CLANG_OFFLOAD_BUNDLE____START__ host-x86_64-unknown-linux-gnu
# 1 "/tmp/test-963ab6.cpp"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 404 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "/tmp/test-header-08e0d3.h" 1
# 2 "<built-in>" 2
# 1 "/tmp/test-963ab6.cpp" 2
# 1 "test.cpp"
int main() { return 0; }
# 1 "/tmp/test-footer-cd3679.h" 1
# 4 "test.cpp" 2

Here, the files /tmp/test-963ab6.cpp, /tmp/test-header-08e0d3.h, /tmp/test-963ab6.cpp and /tmp/test-footer-cd3679.h have randomly generated names that change with each compiler invocation, causing the cache to be invalidated every time.

I assume this is a regression as according to #1797 ccache should work with DPC++.

NOTE: This can be worked around by using ccache's "depend mode" (CCACHE_DEPEND=1), which does not parse preprocessor output to determine dependencies, relying on compiler-generated dependency files instead (-MD/-MMD).

To Reproduce

Create file test.cpp:

int main() { return 0; } 

Run DPC++ with ccache (at least twice):

CCACHE_LOGFILE="$PWD/ccache.log" ccache clang++ -fsycl test.cpp -c -o test.o

Examining the produced ccache.log file will reveal something like this:

...
[2022-01-05T11:51:30.485895 3877683] No b8b4653v0ceorrbv3ra96m7q5m0hd13he in primary storage
[2022-01-05T11:51:30.485942 3877683] Running preprocessor
[2022-01-05T11:51:30.485948 3877683] Executing clang++ -fsycl -E test.cpp
[2022-01-05T11:51:30.551676 3877683] Failed to stat /tmp/test-3f740e.cpp: No such file or directory
[2022-01-05T11:51:30.551697 3877683] Disabling direct mode
...

As you can see, not only are the random names problematic, ccache is additionally unable to access the files which presumably get deleted right away. I'm therefore wondering whether there is any advantage (for tooling) in emitting these file names into the preprocessor output in the first place, or whether they could simply be removed.

Environment:

  • OS: Ubuntu 20.04
  • DPC++ version: clang version 14.0.0 (https://github.com/intel/llvm ec97c573cdf684aec779d19ac68f1e9470ca0516)
  • ccache version: 4.5.1 (also tested with 3.7.7)

psalz avatar Jan 05 '22 11:01 psalz