heads icon indicating copy to clipboard operation
heads copied to clipboard

WiP: Pass all modules from O2 to Os to optimize for space.

Open tlaurion opened this issue 3 years ago • 27 comments

Adresses @aesrentai suggestion to pass modules optimization from O2 (performance) to Os (space) here: https://github.com/osresearch/heads/issues/590#issuecomment-821788703

Will comment on gains after build succeeds. ~Still impossible as of now to build xx30-flash boards on top of linux 5.10.4......~ EDIT: without kernel networking support, this now passes.

tlaurion avatar Feb 22 '22 00:02 tlaurion

Rebased on master to reuse CircleCI 4.11 fixed cache by https://github.com/osresearch/heads/pull/1122

tlaurion avatar Feb 23 '22 22:02 tlaurion

Checkspots:

  • Builds are happening for a new build. Compare boards hashes.txt between builds to see if reproducibility is more broken then with O2. Should not be the case, since make is now being correctly build, and musl-cross-make is being used to build all modules not coreboot, and coreboot toolchain is reproducible to build coreboot. So checking hashes.txt across builds (between different CircleCI used caches) should point new reproducibility errors.
  • Make sure passing CFLAGS in modules is not removing ./configure passed CFLAGS for built modules. We might want to reuse and expend CFLAGS, not just overwrite them.

Todo:

  • [x] Expose space gains before and after passing 02 -> Os.

  • [ ] Make issues for reproducibility issues.

  • ~Update trace of external commands. We should expect to not have any external calls to Make but the built one from now on.~

    • We now depend on OS locally exposed gawk and make

tlaurion avatar Feb 24 '22 15:02 tlaurion

Not all modules are passed from default O2 to Os as of original commit: user@heads-tests:~/heads$ find modules/ | while read line; do echo $line; grep Os $line; done

modules/libassuan
modules/io386
modules/popt
modules/libusb-compat
modules/msrtools
modules/zlib
	CFLAGS="-Os" \
modules/coreboot
modules/gpg
	CFLAGS="-Os"  \
modules/flashtools
modules/libksba
modules/musl-cross
modules/libusb
modules/libgcrypt
	CFLAGS="-Os"  \
modules/frotz
modules/dropbear
modules/slang
modules/lvm2
	CFLAGS="-Os"  \
modules/gpg2
modules/libhidapi-libusb
	CC="$(heads_cc) -Os" \
modules/u-root
modules/json-c
modules/fbwhiptail
modules/linuxboot
modules/kexec
modules/busybox
	CFLAGS="-Os"  \
modules/npth
modules/tpmtotp
	CFLAGS="-I$(INSTALL)/include -Os" \
modules/linux
modules/hidapi
modules/hotp-verification
modules/mbedtls
modules/nkstorecli
modules/libgpg-error
	CFLAGS="-Os"  \
modules/cairo
	CFLAGS="-DCAIRO_NO_MUTEX=1 -Os"  \
modules/util-linux
	CFLAGS="-Os"  \
modules/flashrom
	CFLAGS="-Os" \
modules/coreboot-blobs
modules/libnitrokey
	-DCMAKE_CXX_FLAGS="-Os -fdata-sections -ffunction-sections -ffile-prefix-map=$(pwd)=heads -gno-record-gcc-switches -D__MUSL__ -I$(INSTALL)/include -L$(INSTALL)/lib " \
modules/newt
modules/pixman
	CFLAGS="-Os"  \
modules/qrencode
modules/purism-blobs
modules/pinentry
modules/cryptsetup
	CFLAGS="-Os" \
modules/cryptsetup2
modules/pciutils
modules/libpng
	CFLAGS="-Os" \

tlaurion avatar Jun 30 '22 15:06 tlaurion

Master at https://github.com/osresearch/heads/commit/2cfa94003b97e37d268c923ed2278e24608b5e48

Comparison is done on downloaded initrd.cpio.xz from CircleCI artifact, doing: xz -d initrd.cpio.xz ; for i in initrd.cpio modules.cpio tools.cpio heads.cpio; do cpio -i < $i; done && find . -type f -ls | sort -r -n -k7

user@heads-tests:/tmp/master$ find . -type f -ls | sort -r -n -k7
   134004  12564 -rw-r--r--   1 user     user     12879360 Jun 28 13:07 ./initrd.cpio
   134108   1592 -rwx------   1 user     user      1627856 Jun 30 12:35 ./bin/lvm
   134231   1136 -rwx------   1 user     user      1160520 Jun 30 12:35 ./lib/libgcrypt.so.20
   134065    896 -rwx------   1 user     user       915328 Jun 30 12:35 ./bin/gpg
   134228    740 -rwx------   1 user     user       757232 Jun 30 12:35 ./lib/libcairo.so.2
   134239    652 -rwx------   1 user     user       666216 Jun 30 12:35 ./lib/libpixman-1.so.0
   134227    584 -rwx------   1 user     user       596544 Jun 30 12:35 ./lib/libc.so
   134053    568 -rwx------   1 user     user       580568 Jun 30 12:35 ./bin/flashrom
   134017    496 -rwx------   1 user     user       507776 Jun 30 12:35 ./bin/busybox
   134229    456 -rwx------   1 user     user       464560 Jun 30 12:35 ./lib/libcryptsetup.so.12
   134144    408 -rwx------   1 user     user       416040 Jun 30 12:35 ./bin/scdaemon
   134066    372 -rwx------   1 user     user       380120 Jun 30 12:35 ./bin/gpg-agent
   134243    332 -rwx------   1 user     user       339304 Jun 30 12:35 ./lib/libtpm.so
   134230    328 -rwx------   1 user     user       333240 Jun 30 12:35 ./lib/libdevmapper.so.1.02
   134235    320 -rwx------   1 user     user       325104 Jun 30 12:35 ./lib/libmbedcrypto.so.0
   134248    300 -rw-------   1 user     user       304272 Jun 30 12:35 ./lib/modules/e1000e.ko
   134226    268 -rwx------   1 user     user       271040 Jun 30 12:35 ./lib/libblkid.so.1
   134234    252 -rwx------   1 user     user       256376 Jun 30 12:35 ./lib/libksba.so.8
   134240    204 -rwx------   1 user     user       207912 Jun 30 12:35 ./lib/libpng16.so.16
   134041    184 -rwx------   1 user     user       184824 Jun 30 12:35 ./bin/dropbear
   134162    176 -rwx------   1 user     user       176408 Jun 30 12:35 ./bin/ssh
   134084    168 -rwx------   1 user     user       170984 Jun 30 12:35 ./bin/kexec
   134252    160 -rw-------   1 user     user       159840 Jun 30 12:35 ./lib/modules/xhci-hcd.ko
   134040    136 -rwx------   1 user     user       137464 Jun 30 12:35 ./bin/dmsetup
   134251    132 -rw-------   1 user     user       132360 Jun 30 12:35 ./lib/modules/usb-storage.ko
   134232    128 -rwx------   1 user     user       130000 Jun 30 12:35 ./lib/libgpg-error.so.0
   134178    124 -rwx------   1 user     user       126584 Jun 30 12:35 ./bin/tpm
   134030    116 -rwx------   1 user     user       115192 Jun 30 12:35 ./bin/cryptsetup
   134247    108 -rwx------   1 user     user       108832 Jun 30 12:35 ./lib/libz.so.1
   134245     96 -rwx------   1 user     user        96896 Jun 30 12:35 ./lib/libusb-1.0.so.0
   134031     88 -rwx------   1 user     user        87968 Jun 30 12:35 ./bin/cryptsetup-reencrypt
   134233     80 -rwx------   1 user     user        81024 Jun 30 12:35 ./lib/libjson-c.so.5
   134225     76 -rwx------   1 user     user        76736 Jun 30 12:35 ./lib/libassuan.so.0
   134105     72 -rwx------   1 user     user        73600 Jun 30 12:35 ./bin/lspci
   134249     64 -rw-------   1 user     user        64800 Jun 30 12:35 ./lib/modules/ehci-hcd.ko
   134238     52 -rwx------   1 user     user        52272 Jun 30 12:35 ./lib/libpci.so.3.5.4
   134237     52 -rwx------   1 user     user        52272 Jun 30 12:35 ./lib/libpci.so.3
   134130     52 -rwx------   1 user     user        52200 Jun 30 12:35 ./bin/pinentry-tty
   134049     52 -rwx------   1 user     user        51616 Jun 30 12:35 ./bin/fbwhiptail
   134241     48 -rwx------   1 user     user        48088 Jun 30 12:35 ./lib/libpopt.so.0
   134242     48 -rwx------   1 user     user        47448 Jun 30 12:35 ./lib/libqrencode.so.3
   134195     48 -rwx------   1 user     user        45264 Jun 30 12:35 ./bin/veritysetup
   134081     32 -rwx------   1 user     user        29944 Jun 30 12:35 ./bin/hotp_verification
   134211     28 -rw-------   1 user     user        27936 Jun 30 12:35 ./etc/distro/keys/tails.key
   134023     28 -rwx------   1 user     user        27048 Jun 30 12:35 ./bin/cbmem
   134123     24 -rwx------   1 user     user        24517 Jun 30 12:35 ./bin/oem-factory-reset
   134145     24 -rwx------   1 user     user        22664 Jun 30 12:35 ./bin/scp
   134055     24 -rwx------   1 user     user        22432 Jun 30 12:35 ./bin/flashtool
   134071     20 -rwx------   1 user     user        18844 Jun 30 12:35 ./bin/gui-init
   134244     20 -rwx------   1 user     user        18464 Jun 30 12:35 ./lib/libusb-0.1.so.4
   134021     20 -rwx------   1 user     user        18352 Jun 30 12:35 ./bin/cbfs
   134183     20 -rwx------   1 user     user        18320 Jun 30 12:35 ./bin/uefi
   134246     16 -rwx------   1 user     user        14656 Jun 30 12:35 ./lib/libuuid.so.1
   134236     16 -rwx------   1 user     user        14552 Jun 30 12:35 ./lib/libnpth.so.0
   134133     16 -rwx------   1 user     user        14200 Jun 30 12:35 ./bin/poke
   134250     12 -rw-------   1 user     user        10728 Jun 30 12:35 ./lib/modules/ehci-pci.ko
   134253     12 -rw-------   1 user     user        10568 Jun 30 12:35 ./lib/modules/xhci-pci.ko
   134127     12 -rwx------   1 user     user        10096 Jun 30 12:35 ./bin/peek
   134012     12 -rwx------   1 user     user        10088 Jun 30 12:35 ./bin/base32
   134176     12 -rwx------   1 user     user        10048 Jun 30 12:35 ./bin/totp
   134079     12 -rwx------   1 user     user        10024 Jun 30 12:35 ./bin/hotp
   134093     12 -rwx------   1 user     user        10022 Jun 30 12:35 ./bin/kexec-select-boot
   134217     12 -rw-------   1 user     user         9734 Jun 30 12:35 ./etc/luks-functions
   134067     12 -rwx------   1 user     user         9003 Jun 30 12:35 ./bin/gpg-gui.sh
   134213     12 -rwx------   1 user     user         8892 Jun 30 12:35 ./etc/functions
   134138      8 -rwx------   1 user     user         5912 Jun 30 12:35 ./bin/qrenc
   134052      8 -rwx------   1 user     user         5268 Jun 30 12:35 ./bin/flash.sh
   134027      8 -rwx------   1 user     user         5030 Jun 30 12:35 ./bin/config-gui.sh
   134092      8 -rwx------   1 user     user         4195 Jun 30 12:35 ./bin/kexec-seal-key
   134222      4 -rwx------   1 user     user         3891 Jun 30 12:35 ./init
   134089      4 -rwx------   1 user     user         3785 Jun 30 12:35 ./bin/kexec-parse-boot
   134116      4 -rwx------   1 user     user         3510 Jun 30 12:35 ./bin/mount-usb
   134090      4 -rwx------   1 user     user         3408 Jun 30 12:35 ./bin/kexec-save-default
   134146      4 -rwx------   1 user     user         3381 Jun 30 12:35 ./bin/seal-hotpkey
   134085      4 -rwx------   1 user     user         3118 Jun 30 12:35 ./bin/kexec-boot
   134210      4 -rw-------   1 user     user         3078 Jun 30 12:35 ./etc/distro/keys/qubes-testing.key
   134051      4 -rwx------   1 user     user         2557 Jun 30 12:35 ./bin/flash-gui.sh
   134086      4 -rwx------   1 user     user         2344 Jun 30 12:35 ./bin/kexec-insert-key
   134124      4 -rwx------   1 user     user         2303 Jun 30 12:35 ./bin/oem-system-info-xx30
   134193      4 -rwx------   1 user     user         2101 Jun 30 12:35 ./bin/usb-scan
   134088      4 -rwx------   1 user     user         2059 Jun 30 12:35 ./bin/kexec-parse-bls
   134147      4 -rwx------   1 user     user         2026 Jun 30 12:35 ./bin/seal-totp
   134094      4 -rwx------   1 user     user         1909 Jun 30 12:35 ./bin/kexec-sign-config
   134215      4 -rwx------   1 user     user         1888 Jun 30 12:35 ./etc/gui_functions
   134188      4 -rwx------   1 user     user         1838 Jun 30 12:35 ./bin/unseal-hotp
   134208      4 -rw-------   1 user     user         1725 Jun 30 12:35 ./etc/distro/keys/archlinux.key
   134091      4 -rwx------   1 user     user         1677 Jun 30 12:35 ./bin/kexec-save-key
   134209      4 -rw-------   1 user     user         1629 Jun 30 12:35 ./etc/distro/keys/qubes-4.key
   134119      4 -rwx------   1 user     user         1596 Jun 30 12:35 ./bin/network-init-recovery
   134087      4 -rwx------   1 user     user         1430 Jun 30 12:35 ./bin/kexec-iso-init
   134254      4 -rwx------   1 user     user         1373 Jun 30 12:35 ./mount-boot
   134063      4 -rwx------   1 user     user         1299 Jun 30 12:35 ./bin/generic-init
   133722      4 -rw-------   1 user     user         1247 Jun 30 12:35 ./.ash_history
   134080      4 -rwx------   1 user     user         1087 Jun 30 12:35 ./bin/hotp_initialize
   134095      4 -rwx------   1 user     user         1044 Jun 30 12:35 ./bin/kexec-unseal-key
   134271      4 -rwx------   1 user     user         1000 Jun 30 12:35 ./sbin/insmod
   134260      4 -rwx------   1 user     user          922 Jun 30 12:35 ./sbin/config-dhcp.sh
   134022      4 -rwx------   1 user     user          799 Jun 30 12:35 ./bin/cbfs-init
   134096      4 -rwx------   1 user     user          770 Jun 30 12:35 ./bin/key-init
   134179      4 -rwx------   1 user     user          694 Jun 30 12:35 ./bin/tpm-reset
   134184      4 -rwx------   1 user     user          661 Jun 30 12:35 ./bin/uefi-init
   134189      4 -rwx------   1 user     user          634 Jun 30 12:35 ./bin/unseal-totp
   134207      4 -rw-------   1 user     user          625 Jun 30 12:35 ./etc/config
   134201      4 -rwx------   1 user     user          574 Jun 30 12:35 ./bin/x230-flash.init
   134168      4 -rwx------   1 user     user          574 Jun 30 12:35 ./bin/t430-flash.init
   134139      4 -rwx------   1 user     user          538 Jun 30 12:35 ./bin/qubes-measure-luks
   134054      4 -rwx------   1 user     user          360 Jun 30 12:35 ./bin/flashrom-kgpe-d16-openbmc.sh
   134199      4 -rwx------   1 user     user          320 Jun 30 12:35 ./bin/wget-measure.sh
   134140      4 -rwx------   1 user     user          258 Jun 30 12:35 ./bin/reboot
   134192      4 -rwx------   1 user     user          220 Jun 30 12:35 ./bin/usb-init
   134134      4 -rwx------   1 user     user          205 Jun 30 12:35 ./bin/poweroff
   134212      4 -rw-------   1 user     user          197 Jun 30 12:35 ./etc/fstab
   134218      4 -rw-------   1 user     user          174 Jun 30 12:35 ./etc/motd
   134068      4 -rwx------   1 user     user          106 Jun 30 12:35 ./bin/gpgv
   134005      4 -rw-------   1 user     user           73 Jun 30 12:35 ./.gnupg/gpg-agent.conf
   134200      4 -rwx------   1 user     user           35 Jun 30 12:35 ./bin/whiptail
   134221      4 -rw-------   1 user     user           27 Jun 30 12:35 ./etc/shells
   134220      4 -rw-------   1 user     user           27 Jun 30 12:35 ./etc/passwd
   134216      4 -rw-------   1 user     user           20 Jun 30 12:35 ./etc/hosts
   134214      4 -rw-------   1 user     user           10 Jun 30 12:35 ./etc/group
   134006      4 -rw-------   1 user     user           10 Jun 30 12:35 ./.gnupg/gpg.conf
   134255      0 -rw-------   1 user     user            0 Jun 30 12:35 ./run/cryptsetup/.placeholder

Current optimization of this PR effect:

user@heads-tests:/tmp/pr$ find . -type f -ls | sort -r -n -k7
   133724  11644 -rw-r--r--   1 user     user     11938816 Jun 30 11:32 ./initrd.cpio
   133821   1384 -rwx------   1 user     user      1414928 Jun 30 12:35 ./bin/lvm
   133948   1032 -rwx------   1 user     user      1054216 Jun 30 12:35 ./lib/libgcrypt.so.20
   133780    896 -rwx------   1 user     user       915328 Jun 30 12:35 ./bin/gpg
   133944    584 -rwx------   1 user     user       596544 Jun 30 12:35 ./lib/libc.so
   133774    568 -rwx------   1 user     user       580568 Jun 30 12:35 ./bin/flashrom
   133956    560 -rwx------   1 user     user       572016 Jun 30 12:35 ./lib/libpixman-1.so.0
   133738    496 -rwx------   1 user     user       507776 Jun 30 12:35 ./bin/busybox
   133945    480 -rwx------   1 user     user       491024 Jun 30 12:35 ./lib/libcairo.so.2
   133946    456 -rwx------   1 user     user       464560 Jun 30 12:35 ./lib/libcryptsetup.so.12
   133857    408 -rwx------   1 user     user       416040 Jun 30 12:35 ./bin/scdaemon
   133781    372 -rwx------   1 user     user       380120 Jun 30 12:35 ./bin/gpg-agent
   133952    320 -rwx------   1 user     user       325104 Jun 30 12:35 ./lib/libmbedcrypto.so.0
   133960    308 -rwx------   1 user     user       314728 Jun 30 12:35 ./lib/libtpm.so
   133965    300 -rw-------   1 user     user       304272 Jun 30 12:35 ./lib/modules/e1000e.ko
   133947    280 -rwx------   1 user     user       284184 Jun 30 12:35 ./lib/libdevmapper.so.1.02
   133951    252 -rwx------   1 user     user       256376 Jun 30 12:35 ./lib/libksba.so.8
   133943    236 -rwx------   1 user     user       238208 Jun 30 12:35 ./lib/libblkid.so.1
   133762    184 -rwx------   1 user     user       184824 Jun 30 12:35 ./bin/dropbear
   133876    176 -rwx------   1 user     user       176408 Jun 30 12:35 ./bin/ssh
   133957    168 -rwx------   1 user     user       171032 Jun 30 12:35 ./lib/libpng16.so.16
   133797    168 -rwx------   1 user     user       170984 Jun 30 12:35 ./bin/kexec
   133969    160 -rw-------   1 user     user       159840 Jun 30 12:35 ./lib/modules/xhci-hcd.ko
   133968    132 -rw-------   1 user     user       132360 Jun 30 12:35 ./lib/modules/usb-storage.ko
   133761    120 -rwx------   1 user     user       121080 Jun 30 12:35 ./bin/dmsetup
   133894    116 -rwx------   1 user     user       118392 Jun 30 12:35 ./bin/tpm
   133751    116 -rwx------   1 user     user       115192 Jun 30 12:35 ./bin/cryptsetup
   133949    108 -rwx------   1 user     user       109520 Jun 30 12:35 ./lib/libgpg-error.so.0
   133962     96 -rwx------   1 user     user        96896 Jun 30 12:35 ./lib/libusb-1.0.so.0
   133752     88 -rwx------   1 user     user        87968 Jun 30 12:35 ./bin/cryptsetup-reencrypt
   133950     80 -rwx------   1 user     user        81024 Jun 30 12:35 ./lib/libjson-c.so.5
   133942     76 -rwx------   1 user     user        76736 Jun 30 12:35 ./lib/libassuan.so.0
   133964     76 -rwx------   1 user     user        76040 Jun 30 12:35 ./lib/libz.so.1
   133818     72 -rwx------   1 user     user        73600 Jun 30 12:35 ./bin/lspci
   133966     64 -rw-------   1 user     user        64800 Jun 30 12:35 ./lib/modules/ehci-hcd.ko
   133955     52 -rwx------   1 user     user        52272 Jun 30 12:35 ./lib/libpci.so.3.5.4
   133954     52 -rwx------   1 user     user        52272 Jun 30 12:35 ./lib/libpci.so.3
   133843     52 -rwx------   1 user     user        52200 Jun 30 12:35 ./bin/pinentry-tty
   133770     52 -rwx------   1 user     user        51616 Jun 30 12:35 ./bin/fbwhiptail
   133958     48 -rwx------   1 user     user        48088 Jun 30 12:35 ./lib/libpopt.so.0
   133959     48 -rwx------   1 user     user        47448 Jun 30 12:35 ./lib/libqrencode.so.3
   133912     48 -rwx------   1 user     user        45264 Jun 30 12:35 ./bin/veritysetup
   133928     28 -rw-------   1 user     user        27936 Jun 30 12:35 ./etc/distro/keys/tails.key
   133744     28 -rwx------   1 user     user        27048 Jun 30 12:35 ./bin/cbmem
   133836     24 -rwx------   1 user     user        24517 Jun 30 12:35 ./bin/oem-factory-reset
   133858     24 -rwx------   1 user     user        22664 Jun 30 12:35 ./bin/scp
   133776     24 -rwx------   1 user     user        22432 Jun 30 12:35 ./bin/flashtool
   133786     20 -rwx------   1 user     user        18844 Jun 30 12:35 ./bin/gui-init
   133961     20 -rwx------   1 user     user        18464 Jun 30 12:35 ./lib/libusb-0.1.so.4
   133742     20 -rwx------   1 user     user        18352 Jun 30 12:35 ./bin/cbfs
   133899     20 -rwx------   1 user     user        18320 Jun 30 12:35 ./bin/uefi
   133963     16 -rwx------   1 user     user        14720 Jun 30 12:35 ./lib/libuuid.so.1
   133953     16 -rwx------   1 user     user        14552 Jun 30 12:35 ./lib/libnpth.so.0
   133846     16 -rwx------   1 user     user        14200 Jun 30 12:35 ./bin/poke
   133967     12 -rw-------   1 user     user        10728 Jun 30 12:35 ./lib/modules/ehci-pci.ko
   133970     12 -rw-------   1 user     user        10568 Jun 30 12:35 ./lib/modules/xhci-pci.ko
   133840     12 -rwx------   1 user     user        10096 Jun 30 12:35 ./bin/peek
   133892     12 -rwx------   1 user     user        10032 Jun 30 12:35 ./bin/totp
   133806     12 -rwx------   1 user     user        10022 Jun 30 12:35 ./bin/kexec-select-boot
   133934     12 -rw-------   1 user     user         9734 Jun 30 12:35 ./etc/luks-functions
   133782     12 -rwx------   1 user     user         9003 Jun 30 12:35 ./bin/gpg-gui.sh
   133930     12 -rwx------   1 user     user         8892 Jun 30 12:35 ./etc/functions
   133794      8 -rwx------   1 user     user         5912 Jun 30 12:35 ./bin/hotp
   133732      8 -rwx------   1 user     user         5912 Jun 30 12:35 ./bin/base32
   133851      8 -rwx------   1 user     user         5904 Jun 30 12:35 ./bin/qrenc
   133773      8 -rwx------   1 user     user         5268 Jun 30 12:35 ./bin/flash.sh
   133748      8 -rwx------   1 user     user         5030 Jun 30 12:35 ./bin/config-gui.sh
   133805      8 -rwx------   1 user     user         4195 Jun 30 12:35 ./bin/kexec-seal-key
   133939      4 -rwx------   1 user     user         3891 Jun 30 12:35 ./init
   133802      4 -rwx------   1 user     user         3785 Jun 30 12:35 ./bin/kexec-parse-boot
   133829      4 -rwx------   1 user     user         3510 Jun 30 12:35 ./bin/mount-usb
   133803      4 -rwx------   1 user     user         3408 Jun 30 12:35 ./bin/kexec-save-default
   133859      4 -rwx------   1 user     user         3381 Jun 30 12:35 ./bin/seal-hotpkey
   133798      4 -rwx------   1 user     user         3118 Jun 30 12:35 ./bin/kexec-boot
   133927      4 -rw-------   1 user     user         3078 Jun 30 12:35 ./etc/distro/keys/qubes-testing.key
   133772      4 -rwx------   1 user     user         2557 Jun 30 12:35 ./bin/flash-gui.sh
   133799      4 -rwx------   1 user     user         2344 Jun 30 12:35 ./bin/kexec-insert-key
   133837      4 -rwx------   1 user     user         2303 Jun 30 12:35 ./bin/oem-system-info-xx30
   133910      4 -rwx------   1 user     user         2101 Jun 30 12:35 ./bin/usb-scan
   133801      4 -rwx------   1 user     user         2059 Jun 30 12:35 ./bin/kexec-parse-bls
   133860      4 -rwx------   1 user     user         2026 Jun 30 12:35 ./bin/seal-totp
   133807      4 -rwx------   1 user     user         1909 Jun 30 12:35 ./bin/kexec-sign-config
   133932      4 -rwx------   1 user     user         1888 Jun 30 12:35 ./etc/gui_functions
   133905      4 -rwx------   1 user     user         1838 Jun 30 12:35 ./bin/unseal-hotp
   133925      4 -rw-------   1 user     user         1725 Jun 30 12:35 ./etc/distro/keys/archlinux.key
   133804      4 -rwx------   1 user     user         1677 Jun 30 12:35 ./bin/kexec-save-key
   133926      4 -rw-------   1 user     user         1629 Jun 30 12:35 ./etc/distro/keys/qubes-4.key
   133832      4 -rwx------   1 user     user         1596 Jun 30 12:35 ./bin/network-init-recovery
   133800      4 -rwx------   1 user     user         1430 Jun 30 12:35 ./bin/kexec-iso-init
   133971      4 -rwx------   1 user     user         1373 Jun 30 12:35 ./mount-boot
   133778      4 -rwx------   1 user     user         1299 Jun 30 12:35 ./bin/generic-init
   133723      4 -rw-------   1 user     user         1247 Jun 30 12:35 ./.ash_history
   133808      4 -rwx------   1 user     user         1044 Jun 30 12:35 ./bin/kexec-unseal-key
   133988      4 -rwx------   1 user     user         1000 Jun 30 12:35 ./sbin/insmod
   133977      4 -rwx------   1 user     user          922 Jun 30 12:35 ./sbin/config-dhcp.sh
   133743      4 -rwx------   1 user     user          799 Jun 30 12:35 ./bin/cbfs-init
   133809      4 -rwx------   1 user     user          770 Jun 30 12:35 ./bin/key-init
   133895      4 -rwx------   1 user     user          694 Jun 30 12:35 ./bin/tpm-reset
   133900      4 -rwx------   1 user     user          661 Jun 30 12:35 ./bin/uefi-init
   133906      4 -rwx------   1 user     user          634 Jun 30 12:35 ./bin/unseal-totp
   133924      4 -rw-------   1 user     user          615 Jun 30 12:35 ./etc/config
   133918      4 -rwx------   1 user     user          574 Jun 30 12:35 ./bin/x230-flash.init
   133882      4 -rwx------   1 user     user          574 Jun 30 12:35 ./bin/t430-flash.init
   133852      4 -rwx------   1 user     user          538 Jun 30 12:35 ./bin/qubes-measure-luks
   133775      4 -rwx------   1 user     user          360 Jun 30 12:35 ./bin/flashrom-kgpe-d16-openbmc.sh
   133916      4 -rwx------   1 user     user          320 Jun 30 12:35 ./bin/wget-measure.sh
   133853      4 -rwx------   1 user     user          258 Jun 30 12:35 ./bin/reboot
   133909      4 -rwx------   1 user     user          220 Jun 30 12:35 ./bin/usb-init
   133847      4 -rwx------   1 user     user          205 Jun 30 12:35 ./bin/poweroff
   133929      4 -rw-------   1 user     user          197 Jun 30 12:35 ./etc/fstab
   133935      4 -rw-------   1 user     user          174 Jun 30 12:35 ./etc/motd
   133783      4 -rwx------   1 user     user          106 Jun 30 12:35 ./bin/gpgv
   133725      4 -rw-------   1 user     user           73 Jun 30 12:35 ./.gnupg/gpg-agent.conf
   133917      4 -rwx------   1 user     user           35 Jun 30 12:35 ./bin/whiptail
   133938      4 -rw-------   1 user     user           27 Jun 30 12:35 ./etc/shells
   133937      4 -rw-------   1 user     user           27 Jun 30 12:35 ./etc/passwd
   133933      4 -rw-------   1 user     user           20 Jun 30 12:35 ./etc/hosts
   133931      4 -rw-------   1 user     user           10 Jun 30 12:35 ./etc/group
   133726      4 -rw-------   1 user     user           10 Jun 30 12:35 ./.gnupg/gpg.conf
   133972      0 -rw-------   1 user     user            0 Jun 30 12:35 ./run/cryptsetup/.placeholder

tlaurion avatar Jun 30 '22 17:06 tlaurion

Adding gpg2 and cryptsetup2

tlaurion avatar Jun 30 '22 17:06 tlaurion

master:

Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           85100 none
cpu_microcode_blob.bin         0x14d80    microcode       26624 none
fallback/ramstage              0x1b600    stage           97676 none
config                         0x33400    raw               786 none
revision                       0x33780    raw               691 none
fallback/dsdt.aml              0x33a80    raw             14615 none
vbt.bin                        0x37400    raw              1433 LZMA (4281 decompressed)
cmos_layout.bin                0x37a00    cmos_layout      1884 none
fallback/postcar               0x381c0    stage           25816 none
fallback/payload               0x3e700    simple elf    7309255 none
(empty)                        0x736f00   null          4361880 none
bootblock                      0xb5fdc0   bootblock       65536 none

Actual PR:

Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           85100 none
cpu_microcode_blob.bin         0x14d80    microcode       26624 none
fallback/ramstage              0x1b600    stage           97707 none
config                         0x33400    raw               791 none
revision                       0x33780    raw               696 none
fallback/dsdt.aml              0x33a80    raw             14615 none
vbt.bin                        0x37400    raw              1433 LZMA (4281 decompressed)
cmos_layout.bin                0x37a00    cmos_layout      1884 none
fallback/postcar               0x381c0    stage           25816 none
fallback/payload               0x3e700    simple elf    6884295 none
(empty)                        0x6cf300   null          4786840 none
bootblock                      0xb5fdc0   bootblock       65536 none

4786840 - 4361880 = 424960 gain of 424kb from this PR alone

tlaurion avatar Jun 30 '22 19:06 tlaurion

Impact on changes in build logs: master vs this pr:

modules/libassuan
modules/io386
modules/popt
modules/libusb-compat
modules/msrtools
modules/zlib : O2 -> Os
modules/coreboot
modules/gpg : -g -O2 -> Os
modules/flashtools
modules/libksba
modules/musl-cross
modules/libusb
modules/libgcrypt : -g -O2 -> Os
modules/frotz
modules/dropbear
modules/slang
modules/lvm2 : O2 -> Os
modules/gpg2
modules/libhidapi-libusb : not applied
modules/u-root
modules/json-c
modules/fbwhiptail
modules/linuxboot
modules/kexec
modules/busybox Some files O2-> Os, some stays O2.
modules/npth
modules/tpmtotp not specified -> Os
modules/linux
modules/hidapi
modules/hotp-verification
modules/mbedtls
modules/nkstorecli
modules/libgpg-error : -g -O2 -> Os
modules/cairo : 03 -> Os
modules/util-linux : -g -O2 -> Os
modules/flashrom Already Os
modules/coreboot-blobs
modules/libnitrokey : Os
modules/pixman : -g -O2 -> Os
modules/qrencode
modules/purism-blobs
modules/pinentry
modules/cryptsetup : -g -O2 -> Os
modules/cryptsetup2 : -g -O2 -> Os
modules/pciutils
modules/libpng : -g -O2 -> Os 

tlaurion avatar Aug 18 '22 16:08 tlaurion

@JonathonHall-Purism @SergiiDmytruk : LGTM, not perfect, but interesting 400kb gain for all boards as observed under https://github.com/osresearch/heads/pull/1121#issuecomment-1171571652

This would permit us to start adding other stuff, including (while needing prioritizing)

  • Less restrictive Kernel version bumps (and additional space gains there, removing things network adapters and other architecture support etc being in defconfig, and getting away of defconfig for kernel, where defconfig for coreboot is still the recommended approach) #1184 shows an additional 300kb gain
  • Thin LVM support under Heads recovery (part of wyng-backup requirements, not official PR yet
  • #1202
  • #1195
  • #1172 have ACM blobs (DRTM) for additional board configurations
  • #1132
  • Other additional features requiring tools or kernel modules addition

Recommendations on this PR? From previous post:

modules/busybox Some files O2-> Os, some stays O2.
modules/libhidapi-libusb : not applied

Where some previous -g -O2 is now being -Os.

tlaurion avatar Aug 24 '22 15:08 tlaurion

The changes look sane to me.

SergiiDmytruk avatar Aug 24 '22 16:08 SergiiDmytruk

@tlaurion Agree the ~400 KB is significant, however GPG key generation during OEM reset now takes much longer - on Mini v2, went from ~4 minutes to ~7 minutes. I agree -Os makes sense as a default otherwise.

The only other thing I can think of that might be noticeably affected is LUKS re-encryption - I have not actually tried this at all yet, not sure if it is typically I/O or CPU bound.

  • How much of the 400 KB comes from libgcrypt? I'm guessing probably libgcrypt contains the critical path affecting key generation, not gpg/gpg2, have not tested though.
  • Any experience to suggest whether/how LUKS re-encryption might be affected?

JonathonHall-Purism avatar Aug 24 '22 18:08 JonathonHall-Purism

@tlaurion Agree the ~400 KB is significant, however GPG key generation during OEM reset now takes much longer - on Mini v2, went from ~4 minutes to ~7 minutes. I agree -Os makes sense as a default otherwise.

@JonathonHall-Purism : Houla. Will test re-ownership and check re-encryption and gpg in-smartcard generation, not sure how this could be related to this change though, since keys are generated inside of the USB Security dongle; I would understand if in-heads generated keys were affected though, and cryptsetup-reencrypt speed being maybe affected.

Re-encryption is both CPU and IO bound, since blocks are being rewritten in direct-io mode from cryptsetup-reencrypt. Will check results on same disk and same USB Security dongle with master and this PR and report back results editing this post.


EDIT: Used the following oem-factory-reset script, ran from usb (mount-usb, chmod +x that script, ran that script) to use ROM binaries but have the same test case between roms, with dates being printed on screen (and basically just inserting date commands between tasks to be measured so that nothing interrupts the tests): oem-factory-reset.txt Basically applying

--- initrd/bin/oem-factory-reset
+++ initrd/bin/oem-factory-reset
@@ -573,11 +573,15 @@
 
 if [ -n "$luks_new_Disk_Recovery_Key_desired" -a -n "$luks_new_Disk_Recovery_Key_passphrase_desired" ]; then
   #Reencryption of disk, disk recovery key and Disk Recovery Key passphrase change is requested
+  date
   luks_reencrypt
+  date
   luks_change_passphrase
 elif [ -n "$luks_new_Disk_Recovery_Key_desired" -a -z "$luks_new_Disk_Recovery_Key_passphrase_desired" ]; then
   #Reencryption of disk was requested but not passphrase change
+  date
   luks_reencrypt
+  date
 elif [ -z "$luks_new_Disk_Recovery_Key_desired" -a -n "$luks_new_Disk_Recovery_Key_passphrase_desired" ]; then
   #Passphrase change is requested without disk reencryption
   luks_change_passphrase
@@ -603,8 +607,9 @@
 
 ## reset the GPG Key
 echo -e "\nResetting GPG Key...\n(this will take around 3 minutes...)\n"
+date
 gpg_key_reset
-
+date
 # parse name of generated key
 GPG_GEN_KEY=`grep -A1 pub /tmp/gpg_card_edit_output | tail -n1 | sed -nr 's/^([ ])*//p'`
 PUBKEY="/tmp/${GPG_GEN_KEY}.asc"
@@ -727,4 +732,4 @@
 unset luks_passphrase_changed
 unset tpm_password_changed
 
-reboot
+#reboot
  • https://output.circle-artifacts.com/output/job/707c35dd-7bac-4e1a-9047-2520ecd5a913/artifacts/0/build/x230-hotp-maximized/heads-x230-hotp-maximized-v5.0.1-175-g9e54fc48.rom from 9e54fc48 (this PR)
    • gpg in-key keypair generation: 7.3 minutes (438 seconds)
    • cryptsetup-reencrypt: 24m43s
  • https://output.circle-artifacts.com/output/job/8b24a0b1-12df-45dc-a1c0-11ff87bf06fb/artifacts/0/build/x230-hotp-maximized/heads-x230-hotp-maximized-v0.2.0-1229-gf7facf0.rom from f7facf0 (Master)
    • gpg in-key keypair generation: 7.45 minutes (447 seconds)
      • @jans23's https://github.com/osresearch/heads/issues/919#issuecomment-741975244 comment (I admit I never measured before, being used to 4096 keys needing ~15 minutes) said keygen was about 3 minutes total. Is that still true for you @jans23?
    • cryptsetup-reencrypt: 24m17s

This means a variation of

  • 3.3 seconds per key generation (there is 3 keypairs generated in the smartcard)
    • This is not significant to me, and in this test, would be to the advantage of this PR (which would not make sense).
  • less then 30 seconds variation on cryptsetup-reencrypt with direct-io enforced on a Critical mx500 250Gb drive.
    • This would need retesting on better drive then the mx500 drive used in this test. Unfortunately, the best speed I normally get is with Evo PRO 860 drives and I do not have any handy as of now. But the speed is in the normal on my side, but the IO is normally the limit here, and this test at 162Mb/s is in the range for those drives on the x230 i5/i7. The CPU is not the bottleneck here, the IO of the drive normally is, where EVO pro 860 are around 220 mb/s (and I have niot witnessed better speeds in x230 tested drives and seen many).

tlaurion avatar Aug 24 '22 18:08 tlaurion

Thanks @tlaurion , I ran several more tests to get some better data. Bottom line is that the variations in GPG key reset seem to be normal and it was just coincidence in the first few tests that the -Os tests happened to be longer. With that in mind, and LUKS re-encrypt confirmed fine, this PR looks good to me.

Screenshot_20220825_115301

(The failed LUKS tests were apparently just due to lack of space for key slot expansion or something on that disk, unrelated to this PR, I took the 970 EVO from the L14 and put it in the mini to continue.)

Thanks for confirming this with me!

JonathonHall-Purism avatar Aug 25 '22 16:08 JonathonHall-Purism

@SergiiDmytruk @JonathonHall-Purism One area of concern is still:

modules/busybox Some files O2-> Os, some stays O2.
modules/libhidapi-libusb : not applied

For busybox, my Make fu is not good enough to understand why some are passing -02 and some ware passing -0s. I will clean the PR to remove modules which have no change with this PR to clear things up and force push again for one last review before merging.

This freed space is important.

tlaurion avatar Aug 25 '22 16:08 tlaurion

Also, just to be clear, hardcoding CFLAGS="-Os" is removing debugging information (no more -g in the table above). To me its a bonus, but we have to keep that in mind for the future (where everything is stripped per Makefile call to copy binaries and libraries to initrd anyway.)

tlaurion avatar Aug 25 '22 16:08 tlaurion

Current state of master:

# Use coreboot.rom, because custom output files might not be processed by cbfstool
"/home/user/heads/build/coreboot-4.13/qemu-coreboot-fbwhiptail-tpm1-hotp/cbfstool" "/home/user/heads/build/coreboot-4.13/qemu-coreboot-fbwhiptail-tpm1-hotp/coreboot.rom" print
FMAP REGION: COREBOOT
Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           22660 none
fallback/ramstage              0x5980     stage           66122 none
fallback/dsdt.aml              0x15c40    raw              6946 none
cmos_layout.bin                0x177c0    cmos_layout       676 none
fallback/postcar               0x17ac0    stage           23256 none
fallback/payload               0x1d600    simple elf    7931475 none
(empty)                        0x7adcc0   null          1892568 none
bootblock                      0x97bdc0   bootblock       16384 none

Current state of this PR per 9e54fc4:

# Use coreboot.rom, because custom output files might not be processed by cbfstool
"/home/user/heads/build/coreboot-4.13/qemu-coreboot-fbwhiptail-tpm1-hotp/cbfstool" "/home/user/heads/build/coreboot-4.13/qemu-coreboot-fbwhiptail-tpm1-hotp/coreboot.rom" print
FMAP REGION: COREBOOT
Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           22660 none
fallback/ramstage              0x5980     stage           66129 none
fallback/dsdt.aml              0x15c40    raw              6946 none
cmos_layout.bin                0x177c0    cmos_layout       676 none
fallback/postcar               0x17ac0    stage           23256 none
fallback/payload               0x1d600    simple elf    7527507 none
(empty)                        0x74b2c0   null          2296536 none
bootblock                      0x97bdc0   bootblock       16384 none

Modules explicitely passing Os in the builds:

user@heads-tests:~/heads$ grep -e "Os " build/log/* | awk -F ":" {'print $1'} | uniq
build/log/cairo.configure.log
build/log/cairo.log
build/log/cryptsetup2.configure.log
build/log/gpg2.configure.log
build/log/gpg2.log
build/log/kexec.log
build/log/libgcrypt.configure.log
build/log/libgcrypt.log
build/log/libgpg-error.configure.log
build/log/libgpg-error.log
build/log/libksba.configure.log
build/log/libksba.log
build/log/libpng.configure.log
build/log/libpng.log
build/log/lvm2.configure.log
build/log/lvm2.log
build/log/pixman.configure.log
build/log/tpmtotp.log
build/log/util-linux.configure.log
build/log/util-linux.log
build/log/zlib.configure.log

Modules still having O2/O3 instead of Os in the build:

user@heads-tests:~/heads$ grep -e "O3 " -e "O2 " build/log/* | awk -F ":" {'print $1'} | uniq
build/log/kexec.log
build/log/libassuan.log
build/log/pinentry.log

tlaurion avatar Aug 26 '22 14:08 tlaurion

Adding

diff --git a/modules/busybox b/modules/busybox
index e011402a..197dcaf7 100644
--- a/modules/busybox
+++ b/modules/busybox
@@ -13,6 +13,7 @@ busybox_config := config/busybox.config
 busybox_output := busybox
 busybox_target := \
        $(CROSS_TOOLS) \
+       CFLAGS="-Os"  \
        $(MAKE_JOBS) \
 
 # Install symlinks when the busybox program is installed
diff --git a/modules/kexec b/modules/kexec
index 19cee214..1ffd350a 100644
--- a/modules/kexec
+++ b/modules/kexec
@@ -8,6 +8,7 @@ kexec_hash := 40623d4321be2865ef9ea2cd6ec998d31dcf93d0f74353cbd3aa06d8821e3e41
 
 kexec_configure := ./configure \
        $(CROSS_TOOLS) \
+       CFLAGS="-Os" \
        --host i386-elf-linux \
        --target x86_64 \
        --prefix="/" \
diff --git a/modules/libassuan b/modules/libassuan
index 111e6625..89054efc 100644
--- a/modules/libassuan
+++ b/modules/libassuan
@@ -7,6 +7,7 @@ libassuan_hash := 91bcb0403866b4e7c4bc1cc52ed4c364a9b5414b3994f718c70303f7f765e7
 
 libassuan_configure := ./configure \
        $(CROSS_TOOLS) \
+       CFLAGS="-Os" \
        --host x86_64-linux-musl \

Results in increased gain:

"/home/user/heads/build/coreboot-4.13/qemu-coreboot-fbwhiptail-tpm1-hotp/cbfstool" "/home/user/heads/build/coreboot-4.13/qemu-coreboot-fbwhiptail-tpm1-hotp/coreboot.rom" print
FMAP REGION: COREBOOT
Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           22668 none
fallback/ramstage              0x5980     stage           66120 none
fallback/dsdt.aml              0x15c00    raw              6946 none
cmos_layout.bin                0x17780    cmos_layout       676 none
fallback/postcar               0x17a80    stage           23256 none
fallback/payload               0x1d5c0    simple elf    7513171 none
(empty)                        0x747a80   null          2310936 none
bootblock                      0x97bdc0   bootblock       16384 none

tlaurion avatar Aug 26 '22 15:08 tlaurion

7931475 - 7513171= 418304 bytes gain. As of now, kexec passes both O2 and Os to builds, need to investigate

kexec.log

O2 line (single one):

/home/user/heads/crossgcc/bin/x86_64-linux-musl-gcc -fdebug-prefix-map=/home/user/heads=heads -gno-record-gcc-switches -D__MUSL__ -isystem /home/user/heads/install/include -L/home/user/heads/install/lib  -fno-zero-initialized-in-bss -mcmodel=large -Os -fno-builtin -ffreestanding -fno-zero-initialized-in-bss -fno-PIC -fno-PIE -fno-stack-protector -O2  -mcmodel=large -I./purgatory/include -I./purgatory/arch/x86_64/include -I./util_lib/include -I./include -Iinclude -I/home/user/heads/crossgcc/bin/../lib/gcc/x86_64-linux-musl/8.3.0/include  -c -o purgatory/sha256.o util_lib/sha256.c
gcc -O2 -Wall -o bin/bin-to-hex util/bin-to-hex.c

Os lines (the rest of the build lines similar): /home/user/heads/crossgcc/bin/x86_64-linux-musl-gcc -fdebug-prefix-map=/home/user/heads=heads -gno-record-gcc-switches -D__MUSL__ -isystem /home/user/heads/install/include -L/home/user/heads/install/lib -Os -fno-strict-aliasing -Wall -Wstrict-prototypes -I./include -I./util_lib/include -Iinclude/ -I./kexec/arch/x86_64/include -c -MD -o kexec/kexec.o kexec/kexec.c

qemu successfully kexec'ed into installed Fedora XFCE, but still not desireable compilation variation. busybox doesn't show passed Os nor O2/O3 flags in compilation either.

Will push other changes to trigger rebuild, but without kexec nor busybox changes first. (but will continue to apply changes locally and trigger builds)

tlaurion avatar Aug 26 '22 15:08 tlaurion

current masters's c56e9d2 x230-hotp-maximized rom CircleCI board build output:

"/root/project/build/coreboot-4.13/x230-hotp-maximized/cbfstool" "/root/project/build/coreboot-4.13/x230-hotp-maximized/coreboot.rom" print
FMAP REGION: COREBOOT
Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           85100 none
cpu_microcode_blob.bin         0x14d80    microcode       26624 none
fallback/ramstage              0x1b600    stage           97675 none
config                         0x33400    raw               786 none
revision                       0x33780    raw               691 none
fallback/dsdt.aml              0x33a80    raw             14615 none
vbt.bin                        0x37400    raw              1433 LZMA (4281 decompressed)
cmos_layout.bin                0x37a00    cmos_layout      1884 none
fallback/postcar               0x381c0    stage           25816 none
fallback/payload               0x3e700    simple elf    7317959 none
(empty)                        0x739100   null          4353176 none
bootblock                      0xb5fdc0   bootblock       65536 none

current PR's 6e9eb41 x230-hotp-maximized rom CircleCI board build output::

"/root/project/build/coreboot-4.13/x230-hotp-maximized/cbfstool" "/root/project/build/coreboot-4.13/x230-hotp-maximized/coreboot.rom" print
FMAP REGION: COREBOOT
Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           85100 none
cpu_microcode_blob.bin         0x14d80    microcode       26624 none
fallback/ramstage              0x1b600    stage           97690 none
config                         0x33400    raw               786 none
revision                       0x33780    raw               691 none
fallback/dsdt.aml              0x33a80    raw             14615 none
vbt.bin                        0x37400    raw              1433 LZMA (4281 decompressed)
cmos_layout.bin                0x37a00    cmos_layout      1884 none
fallback/postcar               0x381c0    stage           25816 none
fallback/payload               0x3e700    simple elf    6876615 none
(empty)                        0x6cd500   null          4794520 none
bootblock                      0xb5fdc0   bootblock       65536 none

Showing gain of 7317959 - 6876615 = 441344 bytes (without kexec nor busybox passing to Os)

tlaurion avatar Aug 26 '22 16:08 tlaurion

Current xz -d initrd.cpio.xz ; for i in initrd.cpio modules.cpio tools.cpio heads.cpio; do cpio -i < $i; done && find . -type f -ls | sort -r -n -k7

  3809174   1384 -rwx------   1 user     user      1414928 Aug 26 12:23 ./bin/lvm
  4070360   1032 -rwx------   1 user     user      1054216 Aug 26 12:23 ./lib/libgcrypt.so.20
  3152169      912 -rw-r--r--   1 user     user          933376 Aug 11 11:35 ./modules.cpio
  3809130    736 -rwx------   1 user     user       751456 Aug 26 12:23 ./bin/gpg
  4070356    584 -rwx------   1 user     user       596544 Aug 26 12:23 ./lib/libc.so
  3809124    568 -rwx------   1 user     user       580568 Aug 26 12:23 ./bin/flashrom
  4070368    560 -rwx------   1 user     user       572016 Aug 26 12:23 ./lib/libpixman-1.so.0
  3809087    496 -rwx------   1 user     user       507776 Aug 26 12:23 ./bin/busybox
  4070357    480 -rwx------   1 user     user       491024 Aug 26 12:23 ./lib/libcairo.so.2
  4070358    396 -rwx------   1 user     user       403152 Aug 26 12:23 ./lib/libcryptsetup.so.12
  3809211    340 -rwx------   1 user     user       346376 Aug 26 12:23 ./bin/scdaemon
  4070364    320 -rwx------   1 user     user       325104 Aug 26 12:23 ./lib/libmbedcrypto.so.0
  4070372    308 -rwx------   1 user     user       314728 Aug 26 12:23 ./lib/libtpm.so
  3809131    308 -rwx------   1 user     user       314584 Aug 26 12:23 ./bin/gpg-agent
  4070359    280 -rwx------   1 user     user       284184 Aug 26 12:23 ./lib/libdevmapper.so.1.02
  4070385    272 -rw-------   1 user     user       276096 Aug 26 12:23 ./lib/modules/xhci-hcd.ko
  4070355    236 -rwx------   1 user     user       238208 Aug 26 12:23 ./lib/libblkid.so.1
  4070378    220 -rw-------   1 user     user       223136 Aug 26 12:23 ./lib/modules/e1000.ko
  3154474    204 -rw-r--r--   1 user     user        208896 Aug 26 11:27 ./heads.cpio
  4070363    196 -rwx------   1 user     user       199016 Aug 26 12:23 ./lib/libksba.so.8
  3809112    184 -rwx------   1 user     user       184824 Aug 26 12:23 ./bin/dropbear
  3809229    176 -rwx------   1 user     user       176408 Aug 26 12:23 ./bin/ssh
  4070369    168 -rwx------   1 user     user       171032 Aug 26 12:23 ./lib/libpng16.so.16
  3809149    152 -rwx------   1 user     user       154568 Aug 26 12:23 ./bin/kexec
  4070384    140 -rw-------   1 user     user       140888 Aug 26 12:23 ./lib/modules/usb-storage.ko
  3809111    120 -rwx------   1 user     user       121080 Aug 26 12:23 ./bin/dmsetup
  3809245    116 -rwx------   1 user     user       118392 Aug 26 12:23 ./bin/tpm
  4070379    116 -rw-------   1 user     user       116912 Aug 26 12:23 ./lib/modules/ehci-hcd.ko
  4070361    108 -rwx------   1 user     user       109520 Aug 26 12:23 ./lib/libgpg-error.so.0
  3809101    104 -rwx------   1 user     user       102904 Aug 26 12:23 ./bin/cryptsetup
  4070374     96 -rwx------   1 user     user        96896 Aug 26 12:23 ./lib/libusb-1.0.so.0
  3809102     84 -rwx------   1 user     user        83872 Aug 26 12:23 ./bin/cryptsetup-reencrypt
  4070362     80 -rwx------   1 user     user        81024 Aug 26 12:23 ./lib/libjson-c.so.5
  4070376     76 -rwx------   1 user     user        76040 Aug 26 12:23 ./lib/libz.so.1
  3809171     72 -rwx------   1 user     user        73600 Aug 26 12:23 ./bin/lspci
  4070381     72 -rw-------   1 user     user        73280 Aug 26 12:23 ./lib/modules/ohci-hcd.ko
  4070354     64 -rwx------   1 user     user        64448 Aug 26 12:23 ./lib/libassuan.so.0
  4070383     60 -rw-------   1 user     user        61320 Aug 26 12:23 ./lib/modules/uhci-hcd.ko
  4070367     52 -rwx------   1 user     user        52272 Aug 26 12:23 ./lib/libpci.so.3.5.4
  4070366     52 -rwx------   1 user     user        52272 Aug 26 12:23 ./lib/libpci.so.3
  3809120     52 -rwx------   1 user     user        51616 Aug 26 12:23 ./bin/fbwhiptail
  4070370     48 -rwx------   1 user     user        48088 Aug 26 12:23 ./lib/libpopt.so.0
  4070371     48 -rwx------   1 user     user        47448 Aug 26 12:23 ./lib/libqrencode.so.3
  3809196     44 -rwx------   1 user     user        44008 Aug 26 12:23 ./bin/pinentry-tty
  3809262     44 -rwx------   1 user     user        41168 Aug 26 12:23 ./bin/veritysetup
  3809146     32 -rwx------   1 user     user        29944 Aug 26 12:23 ./bin/hotp_verification
  4070338     28 -rw-------   1 user     user        27936 Aug 26 12:23 ./etc/distro/keys/tails.key
  3809093     28 -rwx------   1 user     user        27048 Aug 26 12:23 ./bin/cbmem
  3809189     24 -rwx------   1 user     user        24501 Aug 26 12:23 ./bin/oem-factory-reset
  3809212     24 -rwx------   1 user     user        22664 Aug 26 12:23 ./bin/scp
  3809126     24 -rwx------   1 user     user        22432 Aug 26 12:23 ./bin/flashtool
  3809136     20 -rwx------   1 user     user        18766 Aug 26 12:23 ./bin/gui-init
  4070373     20 -rwx------   1 user     user        18464 Aug 26 12:23 ./lib/libusb-0.1.so.4
  3809091     20 -rwx------   1 user     user        18352 Aug 26 12:23 ./bin/cbfs
  3809250     20 -rwx------   1 user     user        18320 Aug 26 12:23 ./bin/uefi
  4070375     16 -rwx------   1 user     user        14720 Aug 26 12:23 ./lib/libuuid.so.1
  4070365     16 -rwx------   1 user     user        14552 Aug 26 12:23 ./lib/libnpth.so.0
  3809206     16 -rwx------   1 user     user        14520 Aug 26 12:23 ./bin/rdmsr
  3809199     16 -rwx------   1 user     user        14200 Aug 26 12:23 ./bin/poke
  4070382     16 -rw-------   1 user     user        13896 Aug 26 12:23 ./lib/modules/ohci-pci.ko
  4070386     16 -rw-------   1 user     user        13448 Aug 26 12:23 ./lib/modules/xhci-pci.ko
  4070380     16 -rw-------   1 user     user        12312 Aug 26 12:23 ./lib/modules/ehci-pci.ko
  3152167     12 -rw-r--r--   1 user     user         11775 Aug 26 12:26 ./hashes.txt
  3809193     12 -rwx------   1 user     user        10096 Aug 26 12:23 ./bin/peek
  3809268     12 -rwx------   1 user     user        10080 Aug 26 12:23 ./bin/wrmsr
  3809100     12 -rwx------   1 user     user        10080 Aug 26 12:23 ./bin/cpuid
  3809243     12 -rwx------   1 user     user        10032 Aug 26 12:23 ./bin/totp

Current applied modules Os optimized for space build options

user@heads-tests:~/heads$ find modules/ | sort | while read line; do echo $line; grep Os $line; done
modules/
grep: modules/: Is a directory
modules/blake2
modules/busybox
modules/cairo
	CFLAGS="-DCAIRO_NO_MUTEX=1 -Os"  \
modules/coreboot
modules/coreboot-blobs
modules/cryptsetup
	CFLAGS="-Os" \
modules/cryptsetup2
	CFLAGS="-Os" \
modules/dropbear
modules/fbwhiptail
modules/flashrom
modules/flashtools
modules/frotz
modules/gpg
	CFLAGS="-Os"  \
modules/gpg2
	CFLAGS="-Os"  \
modules/hidapi
modules/hotp-verification
modules/io386
modules/json-c
modules/kexec
modules/libassuan
	CFLAGS="-Os" \
modules/libgcrypt
	CFLAGS="-Os"  \
modules/libgpg-error
	CFLAGS="-Os"  \
modules/libhidapi-libusb
	CC="$(heads_cc) -Os" \
modules/libksba
	CFLAGS="-Os" \
modules/libnitrokey
	-DCMAKE_CXX_FLAGS="-Os -fdata-sections -ffunction-sections -ffile-prefix-map=$(pwd)=heads -gno-record-gcc-switches -D__MUSL__ -I$(INSTALL)/include -L$(INSTALL)/lib " \
modules/libpng
	CFLAGS="-Os" \
modules/libusb
modules/libusb-compat
modules/linux
modules/linuxboot
modules/lvm2
	CFLAGS="-Os"  \
modules/mbedtls
modules/msrtools
modules/musl-cross
modules/newt
modules/nkstorecli
modules/npth
modules/pciutils
modules/pinentry
	CFLAGS="-Os" \
modules/pixman
	CFLAGS="-Os"  \
modules/popt
modules/purism-blobs
modules/qrencode
modules/slang
modules/tpmtotp
	CFLAGS="-I$(INSTALL)/include -Os" \
modules/u-root
modules/util-linux
	CFLAGS="-Os"  \
modules/zlib
	CFLAGS="-Os" \
modules/zstd

tlaurion avatar Aug 26 '22 16:08 tlaurion

diff --git a/patches/busybox-1.32.0.patch b/patches/busybox-1.32.0.patch
index dddd1875..b7235be7 100644
--- a/patches/busybox-1.32.0.patch
+++ b/patches/busybox-1.32.0.patch
@@ -10,3 +10,17 @@ diff -u --recursive clean/busybox-1.28.0/libbb/messages.c busybox-1.28.0/libbb/m
  #endif
  
  const char bb_banner[] ALIGN1 = "BusyBox v" BB_VER BB_EXTRA_VERSION;
+--- ./Makefile 2022-08-26 12:48:54.646000000 -0400
++++ ./Makefile 2022-08-26 12:49:21.676000000 -0400
+@@ -278,8 +278,8 @@
+ # We need some generic definitions
+ include $(srctree)/scripts/Kbuild.include
+ 
+-HOSTCFLAGS    += $(call hostcc-option,-Wall -Wstrict-prototypes -O2 -fomit-frame-pointer,)
+-HOSTCXXFLAGS  += -O2
++HOSTCFLAGS    += $(call hostcc-option,-Wall -Wstrict-prototypes -Os -fomit-frame-pointer,)
++HOSTCXXFLAGS  += -Os
+ 
+ # For maximum performance (+ possibly random breakage, uncomment
+ # the following)
+

master: 3809087 496 -rwx------ 1 user user 507776 Aug 26 12:23 ./bin/busybox

local: 3809087 496 -rwx------ 1 user user 507776 Aug 26 12:23 ./bin/busybox

Not applying changes to busybox, no gain.

tlaurion avatar Aug 26 '22 16:08 tlaurion

Testcase flawed, timestamps not changed.... No. Final binaries untouched because the same, so not copied over from build dir to initrd.

tlaurion avatar Aug 26 '22 17:08 tlaurion

Master's initrd https://output.circle-artifacts.com/output/job/aca558bc-6294-4ec6-955e-fbd2a987bfb3/artifacts/0/build/x230-hotp-maximized/initrd.cpio.xz:

user@heads-tests:/tmp/master$ find . -type f -ls | sort -r -n -k7
   134700  12564 -rw-r--r--   1 user     user     12879872 Aug 24 19:31 ./initrd.cpio
   134800   1596 -rwx------   1 user     user      1627856 Aug 26 13:28 ./bin/lvm
   134928   1136 -rwx------   1 user     user      1160520 Aug 26 13:28 ./lib/libgcrypt.so.20
   134757    896 -rwx------   1 user     user       915328 Aug 26 13:28 ./bin/gpg
   134925    740 -rwx------   1 user     user       757232 Aug 26 13:28 ./lib/libcairo.so.2
   134936    652 -rwx------   1 user     user       666216 Aug 26 13:28 ./lib/libpixman-1.so.0
   134924    584 -rwx------   1 user     user       596544 Aug 26 13:28 ./lib/libc.so
   134751    568 -rwx------   1 user     user       580568 Aug 26 13:28 ./bin/flashrom
   134715    496 -rwx------   1 user     user       507776 Aug 26 13:28 ./bin/busybox
   134926    456 -rwx------   1 user     user       464560 Aug 26 13:28 ./lib/libcryptsetup.so.12
   134836    408 -rwx------   1 user     user       416040 Aug 26 13:28 ./bin/scdaemon
   134758    372 -rwx------   1 user     user       380120 Aug 26 13:28 ./bin/gpg-agent
   134940    332 -rwx------   1 user     user       339304 Aug 26 13:28 ./lib/libtpm.so
   134927    328 -rwx------   1 user     user       333240 Aug 26 13:28 ./lib/libdevmapper.so.1.02
   134932    320 -rwx------   1 user     user       325104 Aug 26 13:28 ./lib/libmbedcrypto.so.0
   134946    300 -rw-------   1 user     user       304272 Aug 26 13:28 ./lib/modules/e1000e.ko
   134923    268 -rwx------   1 user     user       271040 Aug 26 13:28 ./lib/libblkid.so.1
   134931    252 -rwx------   1 user     user       256376 Aug 26 13:28 ./lib/libksba.so.8
   134937    204 -rwx------   1 user     user       207912 Aug 26 13:28 ./lib/libpng16.so.16
   134739    184 -rwx------   1 user     user       184824 Aug 26 13:28 ./bin/dropbear
   134854    176 -rwx------   1 user     user       176408 Aug 26 13:28 ./bin/ssh
   134776    168 -rwx------   1 user     user       170984 Aug 26 13:28 ./bin/kexec
   134950    160 -rw-------   1 user     user       159840 Aug 26 13:28 ./lib/modules/xhci-hcd.ko
   134738    136 -rwx------   1 user     user       137464 Aug 26 13:28 ./bin/dmsetup
   134949    132 -rw-------   1 user     user       132360 Aug 26 13:28 ./lib/modules/usb-storage.ko
   134929    128 -rwx------   1 user     user       130000 Aug 26 13:28 ./lib/libgpg-error.so.0
   134870    124 -rwx------   1 user     user       126584 Aug 26 13:28 ./bin/tpm
   134728    116 -rwx------   1 user     user       115192 Aug 26 13:28 ./bin/cryptsetup
   134944    108 -rwx------   1 user     user       108832 Aug 26 13:28 ./lib/libz.so.1
   134942     96 -rwx------   1 user     user        96896 Aug 26 13:28 ./lib/libusb-1.0.so.0
   134729     88 -rwx------   1 user     user        87968 Aug 26 13:28 ./bin/cryptsetup-reencrypt
   134930     80 -rwx------   1 user     user        81024 Aug 26 13:28 ./lib/libjson-c.so.5
   134922     76 -rwx------   1 user     user        76736 Aug 26 13:28 ./lib/libassuan.so.0
   134797     72 -rwx------   1 user     user        73600 Aug 26 13:28 ./bin/lspci
   134947     64 -rw-------   1 user     user        64800 Aug 26 13:28 ./lib/modules/ehci-hcd.ko
   134935     52 -rwx------   1 user     user        52272 Aug 26 13:28 ./lib/libpci.so.3.5.4
   134934     52 -rwx------   1 user     user        52272 Aug 26 13:28 ./lib/libpci.so.3
   134822     52 -rwx------   1 user     user        52200 Aug 26 13:28 ./bin/pinentry-tty
   134747     52 -rwx------   1 user     user        51616 Aug 26 13:28 ./bin/fbwhiptail
   134938     48 -rwx------   1 user     user        48088 Aug 26 13:28 ./lib/libpopt.so.0
   134939     48 -rwx------   1 user     user        47448 Aug 26 13:28 ./lib/libqrencode.so.3
   134887     48 -rwx------   1 user     user        45264 Aug 26 13:28 ./bin/veritysetup
   134773     32 -rwx------   1 user     user        29944 Aug 26 13:28 ./bin/hotp_verification
   134907     28 -rw-------   1 user     user        27936 Aug 26 13:28 ./etc/distro/keys/tails.key
   134721     28 -rwx------   1 user     user        27048 Aug 26 13:28 ./bin/cbmem
   134815     24 -rwx------   1 user     user        24517 Aug 26 13:28 ./bin/oem-factory-reset
   134837     24 -rwx------   1 user     user        22664 Aug 26 13:28 ./bin/scp
   134753     24 -rwx------   1 user     user        22432 Aug 26 13:28 ./bin/flashtool
   134763     20 -rwx------   1 user     user        18846 Aug 26 13:28 ./bin/gui-init
   134941     20 -rwx------   1 user     user        18464 Aug 26 13:28 ./lib/libusb-0.1.so.4
   134719     20 -rwx------   1 user     user        18352 Aug 26 13:28 ./bin/cbfs
   134875     20 -rwx------   1 user     user        18320 Aug 26 13:28 ./bin/uefi
   134943     16 -rwx------   1 user     user        14656 Aug 26 13:28 ./lib/libuuid.so.1
   134933     16 -rwx------   1 user     user        14552 Aug 26 13:28 ./lib/libnpth.so.0
   134825     16 -rwx------   1 user     user        14200 Aug 26 13:28 ./bin/poke
   137557     12 -rw-r--r--   1 user     user        11141 Aug 26 13:30 ./master
   134948     12 -rw-------   1 user     user        10728 Aug 26 13:28 ./lib/modules/ehci-pci.ko
   134951     12 -rw-------   1 user     user        10568 Aug 26 13:28 ./lib/modules/xhci-pci.ko
   134819     12 -rwx------   1 user     user        10096 Aug 26 13:28 ./bin/peek
   134710     12 -rwx------   1 user     user        10088 Aug 26 13:28 ./bin/base32
   134868     12 -rwx------   1 user     user        10048 Aug 26 13:28 ./bin/totp
   134771     12 -rwx------   1 user     user        10024 Aug 26 13:28 ./bin/hotp
   134785     12 -rwx------   1 user     user        10022 Aug 26 13:28 ./bin/kexec-select-boot
   134913     12 -rw-------   1 user     user         9734 Aug 26 13:28 ./etc/luks-functions
   134759     12 -rwx------   1 user     user         9003 Aug 26 13:28 ./bin/gpg-gui.sh
   134909     12 -rwx------   1 user     user         8892 Aug 26 13:28 ./etc/functions
   134830      8 -rwx------   1 user     user         5912 Aug 26 13:28 ./bin/qrenc
   134750      8 -rwx------   1 user     user         5268 Aug 26 13:28 ./bin/flash.sh
   134725      8 -rwx------   1 user     user         5030 Aug 26 13:28 ./bin/config-gui.sh
   134918      8 -rwx------   1 user     user         4293 Aug 26 13:28 ./init
   134784      8 -rwx------   1 user     user         4195 Aug 26 13:28 ./bin/kexec-seal-key
   134781      4 -rwx------   1 user     user         3785 Aug 26 13:28 ./bin/kexec-parse-boot
   134808      4 -rwx------   1 user     user         3510 Aug 26 13:28 ./bin/mount-usb
   134782      4 -rwx------   1 user     user         3408 Aug 26 13:28 ./bin/kexec-save-default
   134838      4 -rwx------   1 user     user         3381 Aug 26 13:28 ./bin/seal-hotpkey
   134777      4 -rwx------   1 user     user         3118 Aug 26 13:28 ./bin/kexec-boot
   134906      4 -rw-------   1 user     user         3078 Aug 26 13:28 ./etc/distro/keys/qubes-testing.key
   134749      4 -rwx------   1 user     user         2557 Aug 26 13:28 ./bin/flash-gui.sh
   134778      4 -rwx------   1 user     user         2344 Aug 26 13:28 ./bin/kexec-insert-key
   134816      4 -rwx------   1 user     user         2303 Aug 26 13:28 ./bin/oem-system-info-xx30
   134885      4 -rwx------   1 user     user         2101 Aug 26 13:28 ./bin/usb-scan
   134780      4 -rwx------   1 user     user         2059 Aug 26 13:28 ./bin/kexec-parse-bls
   134839      4 -rwx------   1 user     user         2026 Aug 26 13:28 ./bin/seal-totp
   134911      4 -rwx------   1 user     user         1888 Aug 26 13:28 ./etc/gui_functions
   134786      4 -rwx------   1 user     user         1884 Aug 26 13:28 ./bin/kexec-sign-config
   134880      4 -rwx------   1 user     user         1838 Aug 26 13:28 ./bin/unseal-hotp
   134904      4 -rw-------   1 user     user         1725 Aug 26 13:28 ./etc/distro/keys/archlinux.key
   134783      4 -rwx------   1 user     user         1677 Aug 26 13:28 ./bin/kexec-save-key
   134905      4 -rw-------   1 user     user         1629 Aug 26 13:28 ./etc/distro/keys/qubes-4.key
   134811      4 -rwx------   1 user     user         1596 Aug 26 13:28 ./bin/network-init-recovery
   134779      4 -rwx------   1 user     user         1430 Aug 26 13:28 ./bin/kexec-iso-init
   134952      4 -rwx------   1 user     user         1373 Aug 26 13:28 ./mount-boot
   134755      4 -rwx------   1 user     user         1299 Aug 26 13:28 ./bin/generic-init
   134697      4 -rw-------   1 user     user         1247 Aug 26 13:28 ./.ash_history
   134772      4 -rwx------   1 user     user         1087 Aug 26 13:28 ./bin/hotp_initialize
   134787      4 -rwx------   1 user     user         1044 Aug 26 13:28 ./bin/kexec-unseal-key
   134972      4 -rwx------   1 user     user         1000 Aug 26 13:28 ./sbin/insmod
   134961      4 -rwx------   1 user     user          922 Aug 26 13:28 ./sbin/config-dhcp.sh
   134720      4 -rwx------   1 user     user          799 Aug 26 13:28 ./bin/cbfs-init
   134788      4 -rwx------   1 user     user          770 Aug 26 13:28 ./bin/key-init
   134871      4 -rwx------   1 user     user          694 Aug 26 13:28 ./bin/tpm-reset
   134876      4 -rwx------   1 user     user          661 Aug 26 13:28 ./bin/uefi-init
   134881      4 -rwx------   1 user     user          634 Aug 26 13:28 ./bin/unseal-totp
   134901      4 -rw-------   1 user     user          625 Aug 26 13:28 ./etc/config
   134893      4 -rwx------   1 user     user          574 Aug 26 13:28 ./bin/x230-flash.init
   134860      4 -rwx------   1 user     user          574 Aug 26 13:28 ./bin/t430-flash.init
   134831      4 -rwx------   1 user     user          538 Aug 26 13:28 ./bin/qubes-measure-luks
   134752      4 -rwx------   1 user     user          360 Aug 26 13:28 ./bin/flashrom-kgpe-d16-openbmc.sh
   134891      4 -rwx------   1 user     user          320 Aug 26 13:28 ./bin/wget-measure.sh
   134832      4 -rwx------   1 user     user          258 Aug 26 13:28 ./bin/reboot
   134884      4 -rwx------   1 user     user          220 Aug 26 13:28 ./bin/usb-init
   134826      4 -rwx------   1 user     user          205 Aug 26 13:28 ./bin/poweroff
   134908      4 -rw-------   1 user     user          197 Aug 26 13:28 ./etc/fstab
   134914      4 -rw-------   1 user     user          174 Aug 26 13:28 ./etc/motd
   134760      4 -rwx------   1 user     user          106 Aug 26 13:28 ./bin/gpgv
   134702      4 -rw-------   1 user     user           73 Aug 26 13:28 ./.gnupg/gpg-agent.conf
   134892      4 -rwx------   1 user     user           35 Aug 26 13:28 ./bin/whiptail
   134917      4 -rw-------   1 user     user           27 Aug 26 13:28 ./etc/shells
   134916      4 -rw-------   1 user     user           27 Aug 26 13:28 ./etc/passwd
   134912      4 -rw-------   1 user     user           20 Aug 26 13:28 ./etc/hosts
   134910      4 -rw-------   1 user     user           10 Aug 26 13:28 ./etc/group
   134703      4 -rw-------   1 user     user           10 Aug 26 13:28 ./.gnupg/gpg.conf
   134955      0 -rw-------   1 user     user            0 Aug 26 13:28 ./run/cryptsetup/.placeholder

Pr initrd https://output.circle-artifacts.com/output/job/6f1a5aa2-13a1-441d-82c3-fcf9dd31196b/artifacts/0/build/x230-hotp-maximized/initrd.cpio.xz

user@heads-tests:/tmp/pr$ find . -type f -ls | sort -r -n -k7
   134988  11228 -rw-r--r--   1 user     user     11511808 Aug 26 12:11 ./initrd.cpio
   135088   1384 -rwx------   1 user     user      1414928 Aug 26 13:29 ./bin/lvm
   137437   1036 -rwx------   1 user     user      1054216 Aug 26 13:29 ./lib/libgcrypt.so.20
   135045    740 -rwx------   1 user     user       751456 Aug 26 13:29 ./bin/gpg
   137429    584 -rwx------   1 user     user       596544 Aug 26 13:29 ./lib/libc.so
   135039    568 -rwx------   1 user     user       580568 Aug 26 13:29 ./bin/flashrom
   137453    560 -rwx------   1 user     user       572016 Aug 26 13:29 ./lib/libpixman-1.so.0
   135003    496 -rwx------   1 user     user       507776 Aug 26 13:29 ./bin/busybox
   137431    480 -rwx------   1 user     user       491024 Aug 26 13:29 ./lib/libcairo.so.2
   137433    396 -rwx------   1 user     user       403152 Aug 26 13:29 ./lib/libcryptsetup.so.12
   135124    340 -rwx------   1 user     user       346376 Aug 26 13:29 ./bin/scdaemon
   137445    320 -rwx------   1 user     user       325104 Aug 26 13:29 ./lib/libmbedcrypto.so.0
   137461    308 -rwx------   1 user     user       314728 Aug 26 13:29 ./lib/libtpm.so
   135046    308 -rwx------   1 user     user       314584 Aug 26 13:29 ./bin/gpg-agent
   137473    300 -rw-------   1 user     user       304272 Aug 26 13:29 ./lib/modules/e1000e.ko
   137435    280 -rwx------   1 user     user       284184 Aug 26 13:29 ./lib/libdevmapper.so.1.02
   137427    236 -rwx------   1 user     user       238208 Aug 26 13:29 ./lib/libblkid.so.1
   137443    196 -rwx------   1 user     user       199016 Aug 26 13:29 ./lib/libksba.so.8
   135027    184 -rwx------   1 user     user       184824 Aug 26 13:29 ./bin/dropbear
   137289    176 -rwx------   1 user     user       176408 Aug 26 13:29 ./bin/ssh
   137455    168 -rwx------   1 user     user       171032 Aug 26 13:29 ./lib/libpng16.so.16
   135064    168 -rwx------   1 user     user       170984 Aug 26 13:29 ./bin/kexec
   137481    160 -rw-------   1 user     user       159840 Aug 26 13:29 ./lib/modules/xhci-hcd.ko
   137479    132 -rw-------   1 user     user       132360 Aug 26 13:29 ./lib/modules/usb-storage.ko
   135026    120 -rwx------   1 user     user       121080 Aug 26 13:29 ./bin/dmsetup
   137321    116 -rwx------   1 user     user       118392 Aug 26 13:29 ./bin/tpm
   137439    108 -rwx------   1 user     user       109520 Aug 26 13:29 ./lib/libgpg-error.so.0
   135016    104 -rwx------   1 user     user       102904 Aug 26 13:29 ./bin/cryptsetup
   137465     96 -rwx------   1 user     user        96896 Aug 26 13:29 ./lib/libusb-1.0.so.0
   135017     84 -rwx------   1 user     user        83872 Aug 26 13:29 ./bin/cryptsetup-reencrypt
   137441     80 -rwx------   1 user     user        81024 Aug 26 13:29 ./lib/libjson-c.so.5
   137469     76 -rwx------   1 user     user        76040 Aug 26 13:29 ./lib/libz.so.1
   135085     72 -rwx------   1 user     user        73600 Aug 26 13:29 ./bin/lspci
   137475     64 -rw-------   1 user     user        64800 Aug 26 13:29 ./lib/modules/ehci-hcd.ko
   137425     64 -rwx------   1 user     user        64448 Aug 26 13:29 ./lib/libassuan.so.0
   137451     52 -rwx------   1 user     user        52272 Aug 26 13:29 ./lib/libpci.so.3.5.4
   137449     52 -rwx------   1 user     user        52272 Aug 26 13:29 ./lib/libpci.so.3
   135035     52 -rwx------   1 user     user        51616 Aug 26 13:29 ./bin/fbwhiptail
   137457     48 -rwx------   1 user     user        48088 Aug 26 13:29 ./lib/libpopt.so.0
   137459     48 -rwx------   1 user     user        47448 Aug 26 13:29 ./lib/libqrencode.so.3
   135110     44 -rwx------   1 user     user        44008 Aug 26 13:29 ./bin/pinentry-tty
   137355     44 -rwx------   1 user     user        41168 Aug 26 13:29 ./bin/veritysetup
   135061     32 -rwx------   1 user     user        29944 Aug 26 13:29 ./bin/hotp_verification
   137395     28 -rw-------   1 user     user        27936 Aug 26 13:29 ./etc/distro/keys/tails.key
   135009     28 -rwx------   1 user     user        27048 Aug 26 13:29 ./bin/cbmem
   135103     24 -rwx------   1 user     user        24517 Aug 26 13:29 ./bin/oem-factory-reset
   137255     24 -rwx------   1 user     user        22664 Aug 26 13:29 ./bin/scp
   135041     24 -rwx------   1 user     user        22432 Aug 26 13:29 ./bin/flashtool
   135051     20 -rwx------   1 user     user        18846 Aug 26 13:29 ./bin/gui-init
   137463     20 -rwx------   1 user     user        18464 Aug 26 13:29 ./lib/libusb-0.1.so.4
   135007     20 -rwx------   1 user     user        18352 Aug 26 13:29 ./bin/cbfs
   137331     20 -rwx------   1 user     user        18320 Aug 26 13:29 ./bin/uefi
   137467     16 -rwx------   1 user     user        14720 Aug 26 13:29 ./lib/libuuid.so.1
   137447     16 -rwx------   1 user     user        14552 Aug 26 13:29 ./lib/libnpth.so.0
   135113     16 -rwx------   1 user     user        14200 Aug 26 13:29 ./bin/poke
   137559     12 -rw-r--r--   1 user     user        11137 Aug 26 13:30 ./pr
   137477     12 -rw-------   1 user     user        10728 Aug 26 13:29 ./lib/modules/ehci-pci.ko
   137483     12 -rw-------   1 user     user        10568 Aug 26 13:29 ./lib/modules/xhci-pci.ko
   135107     12 -rwx------   1 user     user        10096 Aug 26 13:29 ./bin/peek
   137317     12 -rwx------   1 user     user        10032 Aug 26 13:29 ./bin/totp
   135073     12 -rwx------   1 user     user        10022 Aug 26 13:29 ./bin/kexec-select-boot
   137407     12 -rw-------   1 user     user         9734 Aug 26 13:29 ./etc/luks-functions
   135047     12 -rwx------   1 user     user         9003 Aug 26 13:29 ./bin/gpg-gui.sh
   137399     12 -rwx------   1 user     user         8892 Aug 26 13:29 ./etc/functions
   135059      8 -rwx------   1 user     user         5912 Aug 26 13:29 ./bin/hotp
   134998      8 -rwx------   1 user     user         5912 Aug 26 13:29 ./bin/base32
   135118      8 -rwx------   1 user     user         5904 Aug 26 13:29 ./bin/qrenc
   135038      8 -rwx------   1 user     user         5268 Aug 26 13:29 ./bin/flash.sh
   135013      8 -rwx------   1 user     user         5030 Aug 26 13:29 ./bin/config-gui.sh
   137417      8 -rwx------   1 user     user         4293 Aug 26 13:29 ./init
   135072      8 -rwx------   1 user     user         4195 Aug 26 13:29 ./bin/kexec-seal-key
   135069      4 -rwx------   1 user     user         3785 Aug 26 13:29 ./bin/kexec-parse-boot
   135096      4 -rwx------   1 user     user         3510 Aug 26 13:29 ./bin/mount-usb
   135070      4 -rwx------   1 user     user         3408 Aug 26 13:29 ./bin/kexec-save-default
   137257      4 -rwx------   1 user     user         3381 Aug 26 13:29 ./bin/seal-hotpkey
   135065      4 -rwx------   1 user     user         3118 Aug 26 13:29 ./bin/kexec-boot
   137393      4 -rw-------   1 user     user         3078 Aug 26 13:29 ./etc/distro/keys/qubes-testing.key
   135037      4 -rwx------   1 user     user         2557 Aug 26 13:29 ./bin/flash-gui.sh
   135066      4 -rwx------   1 user     user         2344 Aug 26 13:29 ./bin/kexec-insert-key
   135104      4 -rwx------   1 user     user         2303 Aug 26 13:29 ./bin/oem-system-info-xx30
   137351      4 -rwx------   1 user     user         2101 Aug 26 13:29 ./bin/usb-scan
   135068      4 -rwx------   1 user     user         2059 Aug 26 13:29 ./bin/kexec-parse-bls
   137259      4 -rwx------   1 user     user         2026 Aug 26 13:29 ./bin/seal-totp
   137403      4 -rwx------   1 user     user         1888 Aug 26 13:29 ./etc/gui_functions
   135074      4 -rwx------   1 user     user         1884 Aug 26 13:29 ./bin/kexec-sign-config
   137341      4 -rwx------   1 user     user         1838 Aug 26 13:29 ./bin/unseal-hotp
   137389      4 -rw-------   1 user     user         1725 Aug 26 13:29 ./etc/distro/keys/archlinux.key
   135071      4 -rwx------   1 user     user         1677 Aug 26 13:29 ./bin/kexec-save-key
   137391      4 -rw-------   1 user     user         1629 Aug 26 13:29 ./etc/distro/keys/qubes-4.key
   135099      4 -rwx------   1 user     user         1596 Aug 26 13:29 ./bin/network-init-recovery
   135067      4 -rwx------   1 user     user         1430 Aug 26 13:29 ./bin/kexec-iso-init
   137485      4 -rwx------   1 user     user         1373 Aug 26 13:29 ./mount-boot
   135043      4 -rwx------   1 user     user         1299 Aug 26 13:29 ./bin/generic-init
   134699      4 -rw-------   1 user     user         1247 Aug 26 13:29 ./.ash_history
   135060      4 -rwx------   1 user     user         1087 Aug 26 13:29 ./bin/hotp_initialize
   135075      4 -rwx------   1 user     user         1044 Aug 26 13:29 ./bin/kexec-unseal-key
   137525      4 -rwx------   1 user     user         1000 Aug 26 13:29 ./sbin/insmod
   137503      4 -rwx------   1 user     user          922 Aug 26 13:29 ./sbin/config-dhcp.sh
   135008      4 -rwx------   1 user     user          799 Aug 26 13:29 ./bin/cbfs-init
   135076      4 -rwx------   1 user     user          770 Aug 26 13:29 ./bin/key-init
   137323      4 -rwx------   1 user     user          694 Aug 26 13:29 ./bin/tpm-reset
   137333      4 -rwx------   1 user     user          661 Aug 26 13:29 ./bin/uefi-init
   137343      4 -rwx------   1 user     user          634 Aug 26 13:29 ./bin/unseal-totp
   137383      4 -rw-------   1 user     user          625 Aug 26 13:29 ./etc/config
   137367      4 -rwx------   1 user     user          574 Aug 26 13:29 ./bin/x230-flash.init
   137301      4 -rwx------   1 user     user          574 Aug 26 13:29 ./bin/t430-flash.init
   135119      4 -rwx------   1 user     user          538 Aug 26 13:29 ./bin/qubes-measure-luks
   135040      4 -rwx------   1 user     user          360 Aug 26 13:29 ./bin/flashrom-kgpe-d16-openbmc.sh
   137363      4 -rwx------   1 user     user          320 Aug 26 13:29 ./bin/wget-measure.sh
   135120      4 -rwx------   1 user     user          258 Aug 26 13:29 ./bin/reboot
   137349      4 -rwx------   1 user     user          220 Aug 26 13:29 ./bin/usb-init
   135114      4 -rwx------   1 user     user          205 Aug 26 13:29 ./bin/poweroff
   137397      4 -rw-------   1 user     user          197 Aug 26 13:29 ./etc/fstab
   137409      4 -rw-------   1 user     user          174 Aug 26 13:29 ./etc/motd
   135048      4 -rwx------   1 user     user          106 Aug 26 13:29 ./bin/gpgv
   134990      4 -rw-------   1 user     user           73 Aug 26 13:29 ./.gnupg/gpg-agent.conf
   137365      4 -rwx------   1 user     user           35 Aug 26 13:29 ./bin/whiptail
   137415      4 -rw-------   1 user     user           27 Aug 26 13:29 ./etc/shells
   137413      4 -rw-------   1 user     user           27 Aug 26 13:29 ./etc/passwd
   137405      4 -rw-------   1 user     user           20 Aug 26 13:29 ./etc/hosts
   137401      4 -rw-------   1 user     user           10 Aug 26 13:29 ./etc/group
   134991      4 -rw-------   1 user     user           10 Aug 26 13:29 ./.gnupg/gpg.conf
   137491      0 -rw-------   1 user     user            0 Aug 26 13:29 ./run/cryptsetup/.placeholder

tlaurion avatar Aug 26 '22 17:08 tlaurion

So basically the next targets would be

   137445    320 -rwx------   1 user     user       325104 Aug 26 13:29 ./lib/libmbedcrypto.so.0
   137461    308 -rwx------   1 user     user       314728 Aug 26 13:29 ./lib/libtpm.so

libttpm already 0s

tlaurion avatar Aug 26 '22 17:08 tlaurion

   137445    320 -rwx------   1 user     user       325104 Aug 26 13:29 ./lib/libmbedcrypto.so.0
  3809178    284 -rwx------   1 user     user       288272 Aug 26 13:47 ./lib/libmbedcrypto.so.0

Gain

tlaurion avatar Aug 26 '22 17:08 tlaurion

Small note: in most modules, $(CROSS_TOOLS) is passed as an environment variable, not passed to ./configure. Unified that in last PR, will report on size gains and if still the same, ready for merge.

Forgot to revert modules/linux to not pass -0s.....

tlaurion avatar Aug 31 '22 16:08 tlaurion

Small note: in most modules, $(CROSS_TOOLS) is passed as an environment variable, not passed to ./configure. Unified that in last PR, will report on size gains and if still the same, ready for merge.

Forgot to revert modules/linux to not pass -0s.....

zlib complains if $(CROSS_TOOLS) not passed as environment variable: https://app.circleci.com/pipelines/github/tlaurion/heads/1184/workflows/a9a87d4a-eae0-424f-8a9f-6a82be3301b3/jobs/10022

It should be passed everywhere as environment variables.

tlaurion avatar Aug 31 '22 18:08 tlaurion

Seems like some modules will omplain if $(CROSS_TOOLS) is not passed as environement variable prior of ./configure, while no impact if all are having it as environement variable.

All passed as environment variable's 04a54d8a's https://app.circleci.com/pipelines/github/tlaurion/heads/1188/workflows/a2fbeca3-e247-4503-bc7d-a7ab3a320df0/jobs/10037

"/root/project/build/x86/coreboot-4.13/x230-hotp-maximized/cbfstool" "/root/project/build/x86/coreboot-4.13/x230-hotp-maximized/coreboot.rom" print
FMAP REGION: COREBOOT
Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           85100 none
cpu_microcode_blob.bin         0x14d80    microcode       26624 none
fallback/ramstage              0x1b600    stage           97689 none
config                         0x33400    raw               834 none
revision                       0x33780    raw               691 none
fallback/dsdt.aml              0x33a80    raw             14615 none
vbt.bin                        0x37400    raw              1433 LZMA (4281 decompressed)
cmos_layout.bin                0x37a00    cmos_layout      1884 none
fallback/postcar               0x381c0    stage           25816 none
fallback/payload               0x3e700    simple elf    6865351 none
(empty)                        0x6ca900   null          4805784 none
bootblock                      0xb5fdc0   bootblock       65536 none
2022-08-31 14:47:34-04:00 INSTALL   build/x86/coreboot-4.13/x230-hotp-maximized/coreboot.rom => build/x86/x230-hotp-maximized/heads-x230-hotp-maximized-v5.0.1-195-g04a54d8a.rom

And non-uniformized's 5a8a29af's https://app.circleci.com/pipelines/github/tlaurion/heads/1187/workflows/27f3f18b-de46-4d5c-824e-f8784164857e/jobs/10033

"/root/project/build/x86/coreboot-4.13/x230-hotp-maximized/cbfstool" "/root/project/build/x86/coreboot-4.13/x230-hotp-maximized/coreboot.rom" print
FMAP REGION: COREBOOT
Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           85100 none
cpu_microcode_blob.bin         0x14d80    microcode       26624 none
fallback/ramstage              0x1b600    stage           97677 none
config                         0x33400    raw               834 none
revision                       0x33780    raw               691 none
fallback/dsdt.aml              0x33a80    raw             14615 none
vbt.bin                        0x37400    raw              1433 LZMA (4281 decompressed)
cmos_layout.bin                0x37a00    cmos_layout      1884 none
fallback/postcar               0x381c0    stage           25816 none
fallback/payload               0x3e700    simple elf    6865351 none
(empty)                        0x6ca900   null          4805784 none
bootblock                      0xb5fdc0   bootblock       65536 none
2022-08-31 14:34:39-04:00 INSTALL   build/x86/coreboot-4.13/x230-hotp-maximized/coreboot.rom => build/x86/x230-hotp-maximized/heads-x230-hotp-maximized-v5.0.1-195-g5a8a29af.rom

Applying uniformization at the same time.

tlaurion avatar Aug 31 '22 20:08 tlaurion

master's f2e45c4:

"/root/project/build/x86/coreboot-4.13/x230-hotp-maximized/cbfstool" "/root/project/build/x86/coreboot-4.13/x230-hotp-maximized/coreboot.rom" print
FMAP REGION: COREBOOT
Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           85100 none
cpu_microcode_blob.bin         0x14d80    microcode       26624 none
fallback/ramstage              0x1b600    stage           97682 none
config                         0x33400    raw               834 none
revision                       0x33780    raw               691 none
fallback/dsdt.aml              0x33a80    raw             14615 none
vbt.bin                        0x37400    raw              1433 LZMA (4281 decompressed)
cmos_layout.bin                0x37a00    cmos_layout      1884 none
fallback/postcar               0x381c0    stage           25816 none
fallback/payload               0x3e700    simple elf    7317959 none
(empty)                        0x739100   null          4353176 none
bootblock                      0xb5fdc0   bootblock       65536 none
2022-09-16 18:02:20-04:00 INSTALL   build/x86/coreboot-4.13/x230-hotp-maximized/coreboot.rom => build/x86/x230-hotp-maximized/heads-x230-hotp-maximized-v0.2.0-1263-gf2e45c4.rom

585e581

"/root/project/build/x86/coreboot-4.13/x230-hotp-maximized/cbfstool" "/root/project/build/x86/coreboot-4.13/x230-hotp-maximized/coreboot.rom" print
FMAP REGION: COREBOOT
Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           85100 none
cpu_microcode_blob.bin         0x14d80    microcode       26624 none
fallback/ramstage              0x1b600    stage           97672 none
config                         0x333c0    raw               834 none
revision                       0x33740    raw               691 none
fallback/dsdt.aml              0x33a40    raw             14615 none
vbt.bin                        0x373c0    raw              1433 LZMA (4281 decompressed)
cmos_layout.bin                0x379c0    cmos_layout      1884 none
fallback/postcar               0x38180    stage           25816 none
fallback/payload               0x3e6c0    simple elf    6862791 none
(empty)                        0x6c9ec0   null          4808408 none
bootblock                      0xb5fdc0   bootblock       65536 none
2022-09-26 13:39:14-04:00 INSTALL   build/x86/coreboot-4.13/x230-hotp-maximized/coreboot.rom => build/x86/x230-hotp-maximized/heads-x230-hotp-maximized-v5.0.1-212-g585e581d.rom

7317959-6862791= 455168 bytes gain

tlaurion avatar Sep 26 '22 18:09 tlaurion

Testing of Talos II server images also worked.

tlaurion avatar Sep 26 '22 20:09 tlaurion

find ./ -name "*.log" | grep -v configure | while read file; do echo "-----$file------"; echo "Os:" $(grep "\-Os" $file | wc -l); echo "O2:" $(grep "\-O2" $file| wc -l); echo "O3:" $(grep "\-O3" $file | wc -l);done
-----./qrencode.log------
Os: 0
O2: 20
O3: 0
-----./linux.log------
Os: 1917
O2: 41
O3: 3
-----./gpg2.log------
Os: 319
O2: 0
O3: 0
-----./coreboot-blobs.log------
Os: 0
O2: 0
O3: 0
-----./zlib.log------
Os: 31
O2: 0
O3: 0
-----./pixman.log------
Os: 222
O2: 0
O3: 0
-----./cryptsetup2.log------
Os: 147
O2: 0
O3: 13
-----./dropbear.log------
Os: 478
O2: 0
O3: 0
-----./tpmtotp.log------
Os: 79
O2: 0
O3: 72
-----./mbedtls.log------
Os: 1
O2: 0
O3: 0
-----./flashtools.log------
Os: 0
O2: 0
O3: 0
-----./pciutils.log------
Os: 0
O2: 27
O3: 0
-----./coreboot.log------
Os: 772
O2: 0
O3: 0
-----./hidapi.log------
Os: 0
O2: 0
O3: 0
-----./hotp-verification.log------
Os: 0
O2: 0
O3: 0
-----./libusb.log------
Os: 0
O2: 32
O3: 0
-----./fbwhiptail.log------
Os: 0
O2: 0
O3: 0
-----./npth.log------
Os: 0
O2: 15
O3: 0
-----./libksba.log------
Os: 65
O2: 0
O3: 0
-----./libgcrypt.log------
Os: 395
O2: 0
O3: 0
-----./popt.log------
Os: 0
O2: 25
O3: 0
-----./libusb-compat.log------
Os: 0
O2: 5
O3: 0
-----./kexec.log------
Os: 10
O2: 42
O3: 0
-----./flashrom.log------
Os: 51
O2: 0
O3: 0
-----./json-c.log------
Os: 0
O2: 0
O3: 0
-----./util-linux.log------
Os: 386
O2: 0
O3: 0
-----./libgpg-error.log------
Os: 46
O2: 0
O3: 0
-----./musl-cross.log------
Os: 0
O2: 14
O3: 0
-----./libpng.log------
Os: 92
O2: 0
O3: 0
-----./busybox.log------
Os: 354
O2: 3
O3: 0
-----./pinentry.log------
Os: 8
O2: 0
O3: 0
-----./libassuan.log------
Os: 60
O2: 0
O3: 0
-----./cairo.log------
Os: 945
O2: 0
O3: 0
-----./lvm2.log------
Os: 173
O2: 0
O3: 0

tlaurion avatar Sep 26 '22 21:09 tlaurion