aura icon indicating copy to clipboard operation
aura copied to clipboard

Feature Request: makeopts for faster compiling

Open internethering opened this issue 2 years ago • 9 comments

Default make is using only one thread, but normaly CPUs have more than one. Please add support for MAKEOPS similiar to gentoo: https://wiki.gentoo.org/wiki//etc/portage/make.conf#MAKEOPTS

internethering avatar Mar 14 '23 02:03 internethering

This depends on the package being built, actually. I have considered adding general parallelism across packages, but I'm concerned that may cause surprises when each instance of makepkg wants access to all cores at the same time.

fosskers avatar Mar 14 '23 02:03 fosskers

I'm running 'aura -Au' since 2 days (18 packages), and I see only one gcc-process at any time. The makepkg aren't running parallel?

internethering avatar Mar 14 '23 16:03 internethering

Some specific packages do use make -j etc. within their PKGBUILD to use more than one core while building. It's not really possible to know ahead of time which packages do this, so I haven't yet implemented parallel package building on Aura's end yet.

fosskers avatar Mar 15 '23 02:03 fosskers

Some specific packages do use make -j etc. within their PKGBUILD to use more than one core while building. It's not really possible to know ahead of time which packages do this, so I haven't yet implemented parallel package building on Aura's end yet.

this is not logical, cause the package maintainer doesn't know how many cores the target pc have. It would be better to have the possibility to overwrite this global

internethering avatar Mar 15 '23 16:03 internethering

Some build processes, like Rust's cargo, are able to automatically use as many cores as the machine, without specifying a number on the command line (example: -j4). If such a package were being built in parallel alongside other ones by Aura, it could really slow down the machine.

fosskers avatar Mar 16 '23 00:03 fosskers

I looked into manpage of cargo build"

-j N, --jobs N Number of parallel jobs to run. May also be specified with the build.jobs config value https://doc.rust-lang.org/cargo/reference/config.html. Defaults to the number of logical CPUs. If negative, it sets the maximum number of parallel jobs to the number of logical CPUs plus provided value. Should not be 0

when I set -j4, then it compiles on 4 cores. no slow down ;)

by the way, after three days I interrupted the aura process, cause I needed my old x230 laptop for another tasks. I switched from gentoo to arch because of way too long compile times, but there is for me no advantage, when AUR-builds are running only on one core and 75% of the cpu is idling

internethering avatar Mar 17 '23 09:03 internethering

Three days? What exactly were you building?

fosskers avatar Mar 20 '23 05:03 fosskers

I tried a $ sudo aura -Au

bin2iso              :: 1.9b-5                      -> 1.9b-6
celt                 :: 0.11.3-4                    -> 0.11.3-6
ceph-libs            :: 15.2.17-1                   -> 17.2.5-6
gnome-icon-theme     :: 3.12.0-6                    -> 3.12.0-7
libglade             :: 2.6.4-7                     -> 2.6.4-8
libkipi              :: 21.12.3-1                   -> 22.04.0-1
linux-pf-docs-git    :: 5.11.6.r156.g4dff02f19fec-1 -> 6.3.r1169532.cda647827052-1
linux-pf-git         :: 5.11.6.r156.g4dff02f19fec-1 -> 6.3.r1169532.cda647827052-1
linux-pf-headers-git :: 5.11.6.r156.g4dff02f19fec-1 -> 6.3.r1169532.cda647827052-1
metis                :: 5.1.0.p10-2                 -> 5.1.0.p11-1
pygtk                :: 2.24.0-8                    -> 2.24.0-12
python2              :: 2.7.18-5                    -> 2.7.18-8
python2-gobject2     :: 2.28.7-6                    -> 2.28.7-7
qt5-webkit           :: 5.212.0alpha4-16            -> 5.212.0alpha4-18
spotify              :: 1:1.1.10.546-4              -> 1:1.1.84.716-13
stellarium           :: 0.22.0-1                    -> 1.2-1
thinkfan             :: 1.0.2-2                     -> 1.3.1-1

internethering avatar Mar 21 '23 05:03 internethering

Ah you're building a custom kernel eh. Are you using Aura 3 (Haskell) or Aura 4 (Rust)?

fosskers avatar Mar 23 '23 00:03 fosskers

The original request, to enable parallel builds, is something I've decided not to do. This is due to the reason stated above: it's generally not possible to know how many cores a single build will use, since they all build in different ways. If multiple makepkg runs were running in parallel, also using multiple cores, this could cause chaos.

fosskers avatar Jun 27 '24 23:06 fosskers