pyrex icon indicating copy to clipboard operation
pyrex copied to clipboard

[RFC] pyrex.ini: add /tmp to default [run]bind

Open rtollert opened this issue 5 months ago • 0 comments

I observed anomalously high disk I/O on my root filesystem during an OpenEmbedded build which used pyrex. (In fact, no rootfs I/O should generally be happening on this machine during an OpenEmbedded build; both the sources and TMPDIR are on other disks, and /tmp is a tmpfs.) After some judicious debugging with fatrace(1) I noticed that virtually all of the I/O was happening against, e.g., /var/lib/docker/overlay2/dc702bbbb061fbbf13b9d06c36d8ccee398257070e6c0455ff209b656b2db2f3/diff/tmp/ccuwDoBf.o.

It appears that under pyrex, bitbake will use the /tmp in the container overlay, not the system's /tmp. I believe that this is a mistake: pyrex containers should always use /tmp directly through a bind mount. My reasoning is as follows.

  • On systems where /tmp is a tmpfs, using it will effect a major performance improvement, and potentially reduce root filesystem wear.
  • By using a tmpfs instead of the root filesystem, this change cannot introduce any OOM that wouldn't have happened anyway without pyrex.
  • The use of /tmp in recipes must already be robust in the presence of parallel builds; any collision between simultaneous builds under different containers represents a fault in the recipe, not in the bind-mounting.
  • No security problems can occur through bind-mounting /tmp that would not have happened anyway when building outside of pyrex.

I tested this change in my own OE build by editing args, not bind; I'm marking this as an RFC insofar as I have not literally tested this change yet.

rtollert avatar Sep 06 '24 17:09 rtollert