robotnix icon indicating copy to clipboard operation
robotnix copied to clipboard

Add option to disable LTO in Kernel build

Open Atemu opened this issue 5 years ago • 7 comments

You need >20GiB of physical RAM for it to complete and even then it takes pretty long.

The extreme memory usage should probably also be documented since the current readme states that 16GB is enough.

Atemu avatar Oct 25 '20 16:10 Atemu

Could you remind me which device/flavor you were having issues with? Were you able to complete a build with enough zram, even with only 16GB of physical ram?

danielfullmer avatar Oct 25 '20 18:10 danielfullmer

Sunfish.

It did not complete with 8GB og zram swap which is quite a lot already IMO.

Atemu avatar Oct 25 '20 22:10 Atemu

So, for vanilla builds, it should be possible to revert to the prebuilt kernel using kernel.useCustom = false;. However, this kernel is out-of-date and likely has security issues. Upcoming GrapheneOS support for sunfish (https://github.com/danielfullmer/robotnix/pull/56) also looks like it has optional prebuilt kernels, which are more likely to be up-to-date in the future. It's currently untested, but I expect kernel.useCustom = false; would work for GrapheneOS as well.

The only other option I can see would be to use robotnix-built kernels from a nix cache, if/when we ever publish them (https://github.com/danielfullmer/robotnix/issues/17).

danielfullmer avatar Oct 26 '20 17:10 danielfullmer

Is there really no way to disable LTO?

All the other steps are fine.

Atemu avatar Oct 26 '20 18:10 Atemu

There are, for example, some scripts in the kernel source dir like: https://github.com/GrapheneOS/kernel_google_sunfish/blob/501c771385ff1ceac889fb902feeba196368a00c/build.config.sunfish_no-cfi

that look like the disable things like CONFIG_LTO, CONFIG_CFI, etc. But you'd have to track down how we could make something like that work in the robotnix kernel module. I haven't looked into it much at all so far.

danielfullmer avatar Oct 26 '20 19:10 danielfullmer

Added LTO+CFI memory usage note here: 6c6524dab08b3d16c2c5189c1bb03dee269becae

danielfullmer avatar Oct 27 '20 02:10 danielfullmer

I was able to build a sunfish kernel with 16GB RAM using ~13GiB ZSTD zram swap. Total memory usage of the linker spiked at 18GiB.

That data compressed >4:1, what the hell is the linker doing with all that memory?

Atemu avatar Nov 19 '20 09:11 Atemu