pyrex icon indicating copy to clipboard operation
pyrex copied to clipboard

pyrex.ini: add /tmp to default [run]bind

Open rtollert opened this issue 1 year ago • 7 comments

I observed anomalously high disk I/O on my root filesystem during an OpenEmbedded build which used pyrex. (In fact, no rootfs I/O should generally be happening on this machine during an OpenEmbedded build; both the sources and TMPDIR are on other disks, and /tmp is a tmpfs.) After some judicious debugging with fatrace(1) I noticed that virtually all of the I/O was happening against, e.g., /var/lib/docker/overlay2/dc702bbbb061fbbf13b9d06c36d8ccee398257070e6c0455ff209b656b2db2f3/diff/tmp/ccuwDoBf.o.

It appears that under pyrex, bitbake will use the /tmp in the container overlay, not the system's /tmp. I believe that this is a mistake: pyrex containers should always use /tmp directly through a bind mount. My reasoning is as follows.

  • On systems where /tmp is a tmpfs, using it will effect a major performance improvement, and potentially reduce root filesystem wear.
  • By using a tmpfs instead of the root filesystem, this change cannot introduce any OOM that wouldn't have happened anyway without pyrex.
  • The use of /tmp in recipes must already be robust in the presence of parallel builds; any collision between simultaneous builds under different containers represents a fault in the recipe, not in the bind-mounting.
  • No security problems can occur through bind-mounting /tmp that would not have happened anyway when building outside of pyrex.

I tested this change in my own OE build by editing args, not bind; I'm marking this as an RFC insofar as I have not literally tested this change yet.

rtollert avatar Sep 06 '24 17:09 rtollert

Ya, I think that makes sense. Please update with an non RFC PR

JoshuaWatt avatar Dec 04 '24 16:12 JoshuaWatt

I guess I can do that :)

JoshuaWatt avatar Dec 04 '24 16:12 JoshuaWatt

Bump. Hmm, what's with this?

check Expected — Waiting for status to be reported

rtollert avatar Apr 14 '25 19:04 rtollert

I think if a PR waits too long to be approved for CI, it won't run CI even once it is approved.... pretty sure this is some github quirk. I think if you rebase your branch and push it again I can approve it and it will run, or maybe even amend the top commit to change the date and push again might work.

JoshuaWatt avatar Apr 15 '25 14:04 JoshuaWatt

I think if a PR waits too long to be approved for CI, it won't run CI even once it is approved.... pretty sure this is some github quirk. I think if you rebase your branch and push it again I can approve it and it will run, or maybe even amend the top commit to change the date and push again might work.

okee, I just rebased and pushed.

rtollert avatar Apr 17 '25 17:04 rtollert

Ok, we had some trouble with our CI, which is now fixed. I did make #105 to try and merge your change without you needing to rebase again, but it appears that there is actually a problem with this change that breaks the CI

JoshuaWatt avatar Apr 17 '25 20:04 JoshuaWatt

Re-rebased.

rtollert avatar Apr 24 '25 22:04 rtollert