bazelbuild
Move default output root on macOS to `$HOME/Library/Caches`
Description of the feature request:
Forking from #25167 and its implementation in #25205, it was requested during code review that the default output root on macOS be moved from the historically relevant /private/var/tmp to the more conventional $HOME/Library/Caches. This issue serves to track that implementation.
As a relatively new contributor, it's unclear if this is would be an incompatible change, though based on the fact I can't see any incompatible startup options, I am assuming it's either atypical or not supported in the release model. Happy to roll this out in any way a contributor sees fit for it to be.
Which category does this issue belong to?
Core
What underlying problem are you trying to solve with this feature?
No response
Which operating system are you running Bazel on?
macOS 15.3.1
What is the output of bazel info release?
release 8.0.1
If bazel info release returns development version or (@non-git), tell us how you built Bazel.
No response
What's the output of git remote get-url origin; git rev-parse HEAD ?
[email protected]:aaronsky/bazel.git
95bbed03758a7651916d0708a892f5342a0421c5
Have you found anything relevant by searching the web?
No response
Any other information, logs, or outputs that you want to share?
No response
@nglevin internally proposed to instead move the default output root to /tmp (a.k.a. /private/tmp). On macOS, /tmp gets wiped on reboot by default, which keeps keeps the amount of disk space used by Bazel in check.
Apparently, running out of disk space due to Bazel is a problem that many Bazel users at Google have experienced - one reported >200 GB in their output root at one point.
Having the output base deleted sounds like a rather crude form of cleanup. For users that don't use a disk or remote cache, having their output base deleted on reboot could easily be a bad surprise.
@tjgq's work on idle cleanup routines could provide a less drastic solution that also has the same behavior across platforms.
+1, the reason we want this is because we want caches to persist across MacOS upgrades. So wiping it on reboot would be a step in the opposite direction.
@tetromino @nglevin considering the motivations that lead to #25205 getting cherry-picked into 8.4.0, can we revisit the hold on this change for Bazel 9?
@meteorcloudy Do you know if there's anyone else I can reach out to so this can get reviewed? If not for 9.0, 9.1?
Quoting @ashgti from internal bug report:
Some notes on folders macOS uses:
| Path | Permissions | Cleanup |
|---|---|---|
| /tmp | +777 | reboot |
| /var/tmp | +777 | never |
| /Library/Caches | +777 | never |
| ~/Library/Caches | +700 | never |
getconf DARWIN_USER_CACHE_DIR [^1] (like /var/folders/_0/_vrx1th904x4bsf1c0_w4bhw00g80m/C/) |
+700 | reboot |
$TEMPDIR (getconf DARWIN_USER_TEMP_DIR) [^1] (like /var/folders/_0/_vrx1th904x4bsf1c0_w4bhw00g80m/T/) |
+700 | reboot and periodic mtime >3 days |
[^1]: See man confstr on macOS or https://github.com/apple-oss-distributions/Libc/blob/af11da5ca9d527ea2f48bb7efbd0f0f2a4ea4812/gen/confstr.3
End of quote.
Since /var/tmp and ~/Library/Caches have the same cleanup behavior, the only problem with moving from one to the other is needing to adjust various scripts for CI etc.
The fix for @nglevin's issue might be to launch bazel with --user_cache_dir=$(getconf DARWIN_USER_TEMP_DIR) in google's internal launcher script - but we need to investigate whether there's a risk of the dirhelper demon deleting bazel's cache files while bazel is running.
Tl;DR: this is probably ok? If we get complaints, we'll roll the change back.