No space left on device when compiling LLVM

[hlopko]

We are consistently seeing No space left on device (which file exactly hits the error varies between builds, which is expected) in our Ubuntu 20.04 builds of LLVM. We know LLVM is a large codebase. For example my bazel-out is currently 3GB. It should be still far below the disk size in Ubuntu workers, which I'm told have 500GB disks. Any help here would be greatly appreciated :)

Project: http://github.com/google/crubit Example CI run: https://buildkite.com/bazel/crubit/builds/40#0181669a-f666-4d2f-b5a7-15649d6a419a