nix Collect build time statistics

I've often found myself wanting to know how long a derivation took to build. The closest I've found to this has been using stat /nix/var/log/nix/drvs/xx/xx[…]-foo-bar.drv and subtracting the atime from the mtime. This is very brittle, however, and I think it would be nice for nix to record this information somewhere. Does this perhaps already exist? Any thoughts on the idea or implementation?

Dec 02 '17 17:12 lheckemann

Hydra does collect build duration, but how to use it? https://hydra.nixos.org/build/90102549

Duration: | 17s

But we don't know number of cores, parallelism and CPU frequency of builder machine. Maybe user-time collection would be great here, as building is usually CPU intensive, and other subsystems are not very loaded.

Local collector would be also great, for both successful builds and failed ones.

Mar 08 '19 07:03 danbst

Yes, collecting stats like user+sys time is the best suggestion so far. I doubt there's something noticeably better that's easy to do, at least in the way Hydra.nixos.org looks (quite diverse).

EDIT: for good usability, we'd need to think of API exposing such meta-data via binary cache. Then nix build might even show ETA :rainbow:

Mar 08 '19 11:03 vcunat

Some other statistics that might be useful:

max disk usage (might be able to get away with only measuring disk usage at the end, but maybe some build systems delete intermediate files before the end? in that case, we'd need to regularly measure)
- would allow us to warn and/or offer to run GC if there's insufficient space to build something
max memory usage
lines/bytes of build output
- assuming that builds spew out logs at a roughly constant rate, this would let us make a decent guess at an ETA even on machines faster/slower than Hydra

May 13 '20 21:05 Gaelan

I love the last idea! To be a bit more precise, you could actually annotate every output line with a timestamp and then with the benefit of hindsight say "after 10% of the build time the builder hat produced 1000 lines". That would cover the case where for example the actual build is relatively quiet but the install phase spews a lot of information.