pcg-cpp
pcg-cpp copied to clipboard
Published
20 hours ago •
imneme
imneme
Cache misses with ccache when pcg-cpp is included, possibly due to __DATE__ and __TIME__
trafficstars
Minimal example:
#include <pcg_random.hpp>
int main() { return 0; }
Compiling twice gives a cache miss the second time:
$ ccache g++ -c -Ipcg-cpp-0.98.1/include pcg-example.cpp
$ ccache -s
cache directory /home/vedranmiletic/.ccache
primary config /home/vedranmiletic/.ccache/ccache.conf
secondary config (readonly) /etc/ccache.conf
stats updated Thu Feb 6 11:39:36 2020
cache hit (direct) 0
cache hit (preprocessed) 0
cache miss 1
cache hit rate 0.00 %
cleanups performed 0
files in cache 1
cache size 4.1 kB
max cache size 5.0 GB
$ ccache g++ -c -Ipcg-cpp-0.98.1/include pcg-example.cpp
$ ccache -s
cache directory /home/vedranmiletic/.ccache
primary config /home/vedranmiletic/.ccache/ccache.conf
secondary config (readonly) /etc/ccache.conf
stats updated Thu Feb 6 11:39:45 2020
cache hit (direct) 0
cache hit (preprocessed) 0
cache miss 2
cache hit rate 0.00 %
cleanups performed 0
files in cache 2
cache size 8.2 kB
max cache size 5.0 GB
The debug log file shows a different hash each time:
[2020-02-06T11:39:35.920164 14858] === CCACHE 3.7.7 STARTED =========================================
[2020-02-06T11:39:35.920164 14858] Config: (default) base_dir =
[2020-02-06T11:39:35.920164 14858] Config: (default) cache_dir = /home/vedranmiletic/.ccache
[2020-02-06T11:39:35.920164 14858] Config: (default) cache_dir_levels = 2
[2020-02-06T11:39:35.920164 14858] Config: (default) compiler =
[2020-02-06T11:39:35.920164 14858] Config: (default) compiler_check = mtime
[2020-02-06T11:39:35.920164 14858] Config: (default) compression = false
[2020-02-06T11:39:35.920164 14858] Config: (default) compression_level = 6
[2020-02-06T11:39:35.920164 14858] Config: (default) cpp_extension =
[2020-02-06T11:39:35.920164 14858] Config: (default) debug = false
[2020-02-06T11:39:35.920164 14858] Config: (default) depend_mode = false
[2020-02-06T11:39:35.920164 14858] Config: (default) direct_mode = true
[2020-02-06T11:39:35.920164 14858] Config: (default) disable = false
[2020-02-06T11:39:35.920164 14858] Config: (default) extra_files_to_hash =
[2020-02-06T11:39:35.920164 14858] Config: (default) hard_link = false
[2020-02-06T11:39:35.920164 14858] Config: (default) hash_dir = true
[2020-02-06T11:39:35.920164 14858] Config: (default) ignore_headers_in_manifest =
[2020-02-06T11:39:35.920164 14858] Config: (default) keep_comments_cpp = false
[2020-02-06T11:39:35.920164 14858] Config: (default) limit_multiple = 0.8
[2020-02-06T11:39:35.920164 14858] Config: (environment) log_file = /tmp/cache-minimal.debug
[2020-02-06T11:39:35.920164 14858] Config: (default) max_files = 0
[2020-02-06T11:39:35.920164 14858] Config: (default) max_size = 5.0G
[2020-02-06T11:39:35.920164 14858] Config: (default) path =
[2020-02-06T11:39:35.920164 14858] Config: (default) pch_external_checksum = false
[2020-02-06T11:39:35.920164 14858] Config: (default) prefix_command =
[2020-02-06T11:39:35.920164 14858] Config: (default) prefix_command_cpp =
[2020-02-06T11:39:35.920164 14858] Config: (default) read_only = false
[2020-02-06T11:39:35.920164 14858] Config: (default) read_only_direct = false
[2020-02-06T11:39:35.920164 14858] Config: (default) recache = false
[2020-02-06T11:39:35.920164 14858] Config: (default) run_second_cpp = true
[2020-02-06T11:39:35.920164 14858] Config: (default) sloppiness =
[2020-02-06T11:39:35.920164 14858] Config: (default) stats = true
[2020-02-06T11:39:35.920164 14858] Config: (default) temporary_dir =
[2020-02-06T11:39:35.920164 14858] Config: (default) umask =
[2020-02-06T11:39:35.920372 14858] Command line: ccache g++ -c -Ipcg-cpp-0.98.1/include pcg-example.cpp
[2020-02-06T11:39:35.920397 14858] Hostname: hephaestus
[2020-02-06T11:39:35.920420 14858] Working directory: /home/vedranmiletic
[2020-02-06T11:39:35.920478 14858] Source file: pcg-example.cpp
[2020-02-06T11:39:35.920488 14858] Object file: pcg-example.o
[2020-02-06T11:39:35.920510 14858] Trying direct lookup
[2020-02-06T11:39:35.920559 14858] Looking for object file hash in /home/vedranmiletic/.ccache/0/d/82650d7ebf4afae8251b89938339df-280.manifest
[2020-02-06T11:39:35.920575 14858] No such manifest file
[2020-02-06T11:39:35.920584 14858] Did not find object file hash in manifest
[2020-02-06T11:39:35.921046 14858] Running preprocessor
[2020-02-06T11:39:35.921070 14858] Executing /usr/bin/g++ -Ipcg-cpp-0.98.1/include -E pcg-example.cpp
[2020-02-06T11:39:36.015075 14858] Found __DATE__ in pcg-cpp-0.98.1/include/pcg_extras.hpp
[2020-02-06T11:39:36.015109 14858] Found __TIME__ in pcg-cpp-0.98.1/include/pcg_extras.hpp
[2020-02-06T11:39:36.015115 14858] Disabling direct mode
[2020-02-06T11:39:36.015660 14858] Got object file hash from preprocessor
[2020-02-06T11:39:36.015688 14858] Object file /home/vedranmiletic/.ccache/1/8/f93581b7617650b562067786ae323b-1161376.o not in cache
[2020-02-06T11:39:36.015700 14858] Running real compiler
[2020-02-06T11:39:36.015816 14858] Executing /usr/bin/g++ -Ipcg-cpp-0.98.1/include -c -o pcg-example.o pcg-example.cpp
[2020-02-06T11:39:36.535746 14858] Unlink /home/vedranmiletic/.ccache/1/8/f93581b7617650b562067786ae323b-1161376.o.tmp.stdout.hephaestus.14858.mXEZFI
[2020-02-06T11:39:36.535816 14858] Unlink /home/vedranmiletic/.ccache/1/8/f93581b7617650b562067786ae323b-1161376.o.tmp.stderr.hephaestus.14858.o0IuNH
[2020-02-06T11:39:36.535879 14858] Copying pcg-example.o to /home/vedranmiletic/.ccache/1/8/f93581b7617650b562067786ae323b-1161376.o via /home/vedranmiletic/.ccache/1/8/f93581b7617650b562067786ae323b-1161376.o.hephaestus.14858.DV8O0G (uncompressed)
[2020-02-06T11:39:36.535946 14858] Stored in cache: pcg-example.o -> /home/vedranmiletic/.ccache/1/8/f93581b7617650b562067786ae323b-1161376.o (copied)
[2020-02-06T11:39:36.536325 14858] Result: cache miss
[2020-02-06T11:39:36.536348 14858] Acquired lock /home/vedranmiletic/.ccache/1/stats.lock
[2020-02-06T11:39:36.536397 14858] Releasing lock /home/vedranmiletic/.ccache/1/stats.lock
[2020-02-06T11:39:36.536402 14858] Unlink /home/vedranmiletic/.ccache/1/stats.lock
[2020-02-06T11:39:40.335718 14864] === CCACHE 3.7.7 STARTED =========================================
[2020-02-06T11:39:44.999691 14865] === CCACHE 3.7.7 STARTED =========================================
[2020-02-06T11:39:44.999691 14865] Config: (default) base_dir =
[2020-02-06T11:39:44.999691 14865] Config: (default) cache_dir = /home/vedranmiletic/.ccache
[2020-02-06T11:39:44.999691 14865] Config: (default) cache_dir_levels = 2
[2020-02-06T11:39:44.999691 14865] Config: (default) compiler =
[2020-02-06T11:39:44.999691 14865] Config: (default) compiler_check = mtime
[2020-02-06T11:39:44.999691 14865] Config: (default) compression = false
[2020-02-06T11:39:44.999691 14865] Config: (default) compression_level = 6
[2020-02-06T11:39:44.999691 14865] Config: (default) cpp_extension =
[2020-02-06T11:39:44.999691 14865] Config: (default) debug = false
[2020-02-06T11:39:44.999691 14865] Config: (default) depend_mode = false
[2020-02-06T11:39:44.999691 14865] Config: (default) direct_mode = true
[2020-02-06T11:39:44.999691 14865] Config: (default) disable = false
[2020-02-06T11:39:44.999691 14865] Config: (default) extra_files_to_hash =
[2020-02-06T11:39:44.999691 14865] Config: (default) hard_link = false
[2020-02-06T11:39:44.999691 14865] Config: (default) hash_dir = true
[2020-02-06T11:39:44.999691 14865] Config: (default) ignore_headers_in_manifest =
[2020-02-06T11:39:44.999691 14865] Config: (default) keep_comments_cpp = false
[2020-02-06T11:39:44.999691 14865] Config: (default) limit_multiple = 0.8
[2020-02-06T11:39:44.999691 14865] Config: (environment) log_file = /tmp/cache-minimal.debug
[2020-02-06T11:39:44.999691 14865] Config: (default) max_files = 0
[2020-02-06T11:39:44.999691 14865] Config: (/home/vedranmiletic/.ccache/ccache.conf) max_size = 5.0G
[2020-02-06T11:39:44.999691 14865] Config: (default) path =
[2020-02-06T11:39:44.999691 14865] Config: (default) pch_external_checksum = false
[2020-02-06T11:39:44.999691 14865] Config: (default) prefix_command =
[2020-02-06T11:39:44.999691 14865] Config: (default) prefix_command_cpp =
[2020-02-06T11:39:44.999691 14865] Config: (default) read_only = false
[2020-02-06T11:39:44.999691 14865] Config: (default) read_only_direct = false
[2020-02-06T11:39:44.999691 14865] Config: (default) recache = false
[2020-02-06T11:39:44.999691 14865] Config: (default) run_second_cpp = true
[2020-02-06T11:39:44.999691 14865] Config: (default) sloppiness =
[2020-02-06T11:39:44.999691 14865] Config: (default) stats = true
[2020-02-06T11:39:44.999691 14865] Config: (default) temporary_dir =
[2020-02-06T11:39:44.999691 14865] Config: (default) umask =
[2020-02-06T11:39:44.999905 14865] Command line: ccache g++ -c -Ipcg-cpp-0.98.1/include pcg-example.cpp
[2020-02-06T11:39:44.999925 14865] Hostname: hephaestus
[2020-02-06T11:39:44.999947 14865] Working directory: /home/vedranmiletic
[2020-02-06T11:39:44.999995 14865] Source file: pcg-example.cpp
[2020-02-06T11:39:45.000004 14865] Object file: pcg-example.o
[2020-02-06T11:39:45.000023 14865] Trying direct lookup
[2020-02-06T11:39:45.000062 14865] Looking for object file hash in /home/vedranmiletic/.ccache/0/d/82650d7ebf4afae8251b89938339df-280.manifest
[2020-02-06T11:39:45.000082 14865] No such manifest file
[2020-02-06T11:39:45.000090 14865] Did not find object file hash in manifest
[2020-02-06T11:39:45.000210 14865] Running preprocessor
[2020-02-06T11:39:45.000218 14865] Executing /usr/bin/g++ -Ipcg-cpp-0.98.1/include -E pcg-example.cpp
[2020-02-06T11:39:45.097064 14865] Found __DATE__ in pcg-cpp-0.98.1/include/pcg_extras.hpp
[2020-02-06T11:39:45.097099 14865] Found __TIME__ in pcg-cpp-0.98.1/include/pcg_extras.hpp
[2020-02-06T11:39:45.097105 14865] Disabling direct mode
[2020-02-06T11:39:45.097646 14865] Got object file hash from preprocessor
[2020-02-06T11:39:45.097670 14865] Object file /home/vedranmiletic/.ccache/f/1/9c68493f22eafad27987912faa2c81-1161376.o not in cache
[2020-02-06T11:39:45.097682 14865] Running real compiler
[2020-02-06T11:39:45.097803 14865] Executing /usr/bin/g++ -Ipcg-cpp-0.98.1/include -c -o pcg-example.o pcg-example.cpp
[2020-02-06T11:39:45.604277 14865] Unlink /home/vedranmiletic/.ccache/f/1/9c68493f22eafad27987912faa2c81-1161376.o.tmp.stdout.hephaestus.14865.YQFOyc
[2020-02-06T11:39:45.604348 14865] Unlink /home/vedranmiletic/.ccache/f/1/9c68493f22eafad27987912faa2c81-1161376.o.tmp.stderr.hephaestus.14865.la6gad
[2020-02-06T11:39:45.604427 14865] Copying pcg-example.o to /home/vedranmiletic/.ccache/f/1/9c68493f22eafad27987912faa2c81-1161376.o via /home/vedranmiletic/.ccache/f/1/9c68493f22eafad27987912faa2c81-1161376.o.hephaestus.14865.qo8Mbf (uncompressed)
[2020-02-06T11:39:45.604516 14865] Stored in cache: pcg-example.o -> /home/vedranmiletic/.ccache/f/1/9c68493f22eafad27987912faa2c81-1161376.o (copied)
[2020-02-06T11:39:45.605002 14865] Result: cache miss
[2020-02-06T11:39:45.605028 14865] Acquired lock /home/vedranmiletic/.ccache/f/stats.lock
[2020-02-06T11:39:45.605092 14865] Releasing lock /home/vedranmiletic/.ccache/f/stats.lock
[2020-02-06T11:39:45.605099 14865] Unlink /home/vedranmiletic/.ccache/f/stats.lock
[2020-02-06T11:39:46.647536 14871] === CCACHE 3.7.7 STARTED =========================================
I just verified that making struct static_arbitrary_seed optional by adding #ifdef PCG_STATIC_ARBITRARY_SEED and #endif around struct static_arbitrary_seed addresses the issue.
I would also highly appreciate having a way to configure out the static arbitrary seed, since it also breaks precompiled headers.
I agree this should be disabled by default because you have no idea that simply using PCG will reduce the efficiency of compilation caching (which is extremely useful on large codebases). On Arrow C++ this probably has cost us many hours of CI time without us knowing.
This also breaks reproducible builds and generates a warning from clang -Wdate-time.
