nixos-generators
nixos-generators copied to clipboard
Published
20 hours ago •
nix-community
nix-community
qcow not working on aarch64
Hi. Generating qcow/raw not working on aarch64:
cmd:
nix run github:nix-community/nixos-generators -- -f qcow -I nixpkgs=channel:nixos-21.11
aarch64
[...]
copying path '/nix/store/ja4skmlpag15hgqpqd40klv9zx4vf0nw-nixos-system-nixos-21.11.337209.c254b8c915a' to 'local'...
copying channel...
installation finished!
mke2fs 1.46.5 (30-Dec-2021)
Discarding device blocks: done
Creating filesystem with 2096640 4k blocks and 524288 inodes
Filesystem UUID: 4f77ce5e-8d1a-4b9e-bc57-c2b1508b115b
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done
copying staging root to image...
[ 0.000000] Linux version 5.3.0 (nixbld@) (gcc version 9.3.0 (GCC)) #1 Wed Apr 20 08:37:32 UTC 2022
[ 0.000000] memblock address range: 0xffffe9c00000 - 0xffffeffff000
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 25249
[ 0.000000] Kernel command line: mem=100M virtio_mmio.device=316@0x1000000:1
[ 0.000000] Dentry cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[ 0.000000] Inode-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.000000] Memory available: 100764k/102396k RAM
[ 0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] NR_IRQS: 4096
[ 0.000000] lkl: irqs initialized
[ 0.000000] clocksource: lkl: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[ 0.000001] lkl: time and timers initialized (irq2)
[ 0.000047] pid_max: default: 4096 minimum: 301
[ 0.000125] Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[ 0.000138] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[ 0.029174] random: get_random_bytes called from _etext+0xd0f0/0x1652c with crng_init=0
[ 0.046738] printk: console [lkl_console0] enabled
[ 0.046828] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[ 0.046846] xor: automatically using best checksumming function 8regs
[ 0.062418] NET: Registered protocol family 16
[ 0.259901] raid6: int64x8 gen() 1893 MB/s
[ 0.433208] raid6: int64x8 xor() 1233 MB/s
[ 0.606181] raid6: int64x4 gen() 1741 MB/s
[ 0.778727] raid6: int64x4 xor() 1137 MB/s
[ 0.952348] raid6: int64x2 gen() 1246 MB/s
[ 1.125299] raid6: int64x2 xor() 843 MB/s
[ 1.297590] raid6: int64x1 gen() 950 MB/s
[ 1.469170] raid6: int64x1 xor() 565 MB/s
[ 1.469193] raid6: using algorithm int64x8 gen() 1893 MB/s
[ 1.469202] raid6: .... xor() 1233 MB/s, rmw enabled
[ 1.469208] raid6: using intx1 recovery algorithm
[ 3.442981] clocksource: Switched to clocksource lkl
[ 3.617404] NET: Registered protocol family 2
[ 3.692121] tcp_listen_portaddr_hash hash table entries: 256 (order: 0, 4096 bytes, linear)
[ 3.692178] TCP established hash table entries: 1024 (order: 1, 8192 bytes, linear)
[ 3.692209] TCP bind hash table entries: 1024 (order: 1, 8192 bytes, linear)
[ 3.692229] TCP: Hash tables configured (established 1024 bind 1024)
[ 3.707698] UDP hash table entries: 128 (order: 0, 4096 bytes, linear)
[ 3.707739] UDP-Lite hash table entries: 128 (order: 0, 4096 bytes, linear)
[ 3.708103] virtio-mmio: Registering device virtio-mmio.0 at 0x1000000-0x100013b, IRQ 1.
[ 3.710038] workingset: timestamp_bits=62 max_order=16 bucket_order=0
[ 3.714080] SGI XFS with ACLs, security attributes, no debug enabled
[ 3.735068] io scheduler mq-deadline registered
[ 3.735099] io scheduler kyber registered
[ 3.735189] virtio-mmio virtio-mmio.0: Failed to enable 64-bit or 32-bit DMA. Trying to continue, but this might not work.
[ 3.746252] virtio_blk virtio0: [vda] 16777216 512-byte logical blocks (8.59 GB/8.00 GiB)
[ 3.747055] vda: vda1
[ 3.747985] NET: Registered protocol family 10
[ 3.756901] Segment Routing with IPv6
[ 3.756984] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 3.777062] Btrfs loaded, crc32c=crc32c-generic
[ 3.777365] Warning: unable to open an initial console.
[ 3.777406] This architecture does not have kernel memory protection.
[ 3.777416] Run /init as init process
[ 3.793367] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts:
[ 3.967907] random: fast init done
[ 8.896063] random: crng init done
[ 108.865014] reboot: Restarting system
WARNING: Image format was not specified for 'nixos.raw' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
loading kernel modules...
[ 0.879011] Module has invalid ELF structures
[ 0.885062] Module has invalid ELF structures
mounting Nix store...
qemu-system-aarch64: warning: 9p: degraded performance: a reasonable high msize should be chosen on client/guest side (chosen msize is <= 8192). See https://wiki.qemu.org/Documentation/9psetup#msize for details.
mounting host's temporary directory...
starting stage 2 (/nix/store/f8ig4vd29nbw5h34pbdlrang78621lsg-vm-run-stage2)
setting up /etc...
chroot: failed to run command 'systemd-tmpfiles': No such file or directory
updating GRUB 2 menu...
installing the GRUB 2 boot loader on /dev/vda...
Installing for arm64-efi platform.
/nix/store/hb2hlwsnc1z2kf5m2wydj69fsdiv59am-grub-2.06/sbin/grub-install: error: cannot find EFI directory.
/nix/store/szm7f1walapsjwrl65mxrsczf56myv3z-install-grub.pl: installation of GRUB on /dev/vda failed: No such file or directory
[ 5.444090] reboot: Power down
error: builder for '/nix/store/mxq86icp26xfsqlqv833i47dibs0jrks-nixos-disk-image.drv' failed with exit code 1
same command, but on x86:
[...]
[ 1.424212] Run /init as init process
[ 1.428184] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts:
[ 1.553651] random: fast init done
[ 4.746953] random: crng init done
[ 23.959031] reboot: Restarting system
warning: KVM is not available - execution will be slow
Consider installing KVM for hardware-accelerated execution.
If KVM is already installed make sure the kernel module is loaded.
WARNING: Image format was not specified for 'nixos.raw' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
cSeaBIOS (version rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org)
iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+3FF8EFF0+3FEEEFF0 CA00
Booting from ROM...
Probing EDD (edd=off to disable)... ocloading kernel modules...
[ 1.890278] Module has invalid ELF structures
[ 1.962771] Module has invalid ELF structures
[ 2.015097] Module has invalid ELF structures
[ 3.185205] Module has invalid ELF structures
[ 3.233123] Module has invalid ELF structures
[ 3.367143] Module has invalid ELF structures
[ 3.427114] Module has invalid ELF structures
[ 3.478493] Module has invalid ELF structures
[ 3.528338] Module has invalid ELF structures
[ 3.632590] Module has invalid ELF structures
[ 3.680146] Module has invalid ELF structures
[ 3.727518] Module has invalid ELF structures
insmod: can't insert '/nix/store/924sskf52vkp9zy0r95fklj2krlxrnxx-linux-5.10.111-shrunk/lib/modules/5.10.111/kernel/arch/x86/crypto/crc32c-intel.ko.xz': No such device
warning: unable to load /nix/store/924sskf52vkp9zy0r95fklj2krlxrnxx-linux-5.10.111-shrunk/lib/modules/5.10.111/kernel/arch/x86/crypto/crc32c-intel.ko.xz
[ 3.783384] Module has invalid ELF structures
[ 3.828591] Module has invalid ELF structures
[ 4.207037] Module has invalid ELF structures
[ 4.305141] Module has invalid ELF structures
[ 4.390909] Module has invalid ELF structures
[ 4.460524] Module has invalid ELF structures
[ 4.510390] Module has invalid ELF structures
mounting Nix store...
qemu-system-x86_64: warning: 9p: degraded performance: a reasonable high msize should be chosen on client/guest side (chosen msize is <= 8192). See https://wiki.qemu.org/Documentation/9psetup#msize for details.
mounting host's temporary directory...
starting stage 2 (/nix/store/cfgb7sj8rvkzlwp7b7059yw6n2pkyf4c-vm-run-stage2)
setting up /etc...
chroot: failed to run command 'systemd-tmpfiles': No such file or directory
updating GRUB 2 menu...
installing the GRUB 2 boot loader on /dev/vda...
Installing for i386-pc platform.
Installation finished. No error reported.
tune2fs 1.46.5 (30-Dec-2021)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds
Setting time filesystem last checked to Tue Apr 26 05:21:29 2022
[ 32.171519] reboot: Power down
/nix/store/c2k36i36dvmjnkx95ww2j1cf9pbhpfig-nixos-disk-image/nixos.qcow2
yes, grub-install fails on aarch64 since /dev/vda is not available. maybe the primary disk has a different name there? not sure if there is any aarch64 computer which does bios boot anyway. so I guess I need to do EFI in the case of aarch64.
I guess I will go with a check on the current system and if its aarch64 I will try to do EFI by default
maybe I will do a qcow-efi format, since everything would be different basically :D I guess the cleanest way would be to do a module for make-disk-image.nix which exposes all the different arguments via options. so people can just configure in their configuration.nix what they want.
raw-efi is working :)
[ 0.948064] Module has invalid ELF structures
mounting Nix store...
qemu-system-aarch64: warning: 9p: degraded performance: a reasonable high msize should be chosen on client/guest side (chosen msize is <= 8192). See https://wiki.qemu.org/Documentation/9psetup#msize for details.
mounting host's temporary directory...
starting stage 2 (/nix/store/f8ig4vd29nbw5h34pbdlrang78621lsg-vm-run-stage2)
mkfs.fat 4.1 (2017-01-24)
setting up /etc...
chroot: failed to run command 'systemd-tmpfiles': No such file or directory
updating GRUB 2 menu...
installing the GRUB 2 EFI boot loader into /boot...
Installing for arm64-efi platform.
Installation finished. No error reported.
tune2fs 1.46.5 (30-Dec-2021)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds
Setting time filesystem last checked to Tue Apr 26 11:54:26 2022
[ 106.334197] reboot: Power down
/nix/store/ryw74xd9wvyf3yqfkq1bgr7qh93wcxj8-nixos-disk-image/nixos.img
@Lassulus do I need kvm? I tried to build image on cloud instance aarch64, but it is failing.
nix run . -- -f raw-efi -I nixpkgs=channel:nixos-unstable
[...]
[ 1.373692] io scheduler mq-deadline registered
[ 1.373700] io scheduler kyber registered
[ 1.373721] virtio-mmio virtio-mmio.0: Failed to enable 64-bit or 32-bit DMA. Trying to continue, but this might not work.
[ 1.376695] virtio_blk virtio0: [vda] 5689344 512-byte logical blocks (2.91 GB/2.71 GiB)
[ 1.377230] vda: vda1
[ 1.377524] NET: Registered protocol family 10
[ 1.377924] Segment Routing with IPv6
[ 1.377952] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 1.378343] Btrfs loaded, crc32c=crc32c-generic
[ 1.378446] Warning: unable to open an initial console.
[ 1.378487] This architecture does not have kernel memory protection.
[ 1.378494] Run /init as init process
[ 1.379516] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts:
[ 1.445518] random: fast init done
[ 2.139989] random: crng init done
[ 11.539583] reboot: Restarting system
WARNING: Image format was not specified for 'nixos.raw' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
Could not access KVM kernel module: No such file or directory
qemu-system-aarch64: failed to initialize kvm: No such file or directory
qemu-system-aarch64: falling back to tcg
loading kernel modules...
[ 9.272316] Module has invalid ELF structures
[ 9.433105] Module has invalid ELF structures
mounting Nix store...
mounting host's temporary directory...
starting stage 2 (/nix/store/m2wqdxfq14jid18h7pqn1p656i4jhr57-vm-run-stage2)
setting up /etc...
/etc/tmpfiles.d/journal-nocow.conf:26: Failed to resolve specifier: uninitialized /etc detected, skipping
All rules containing unresolvable specifiers will be skipped.
updating GRUB 2 menu...
installing the GRUB 2 EFI boot loader into /boot...
Installing for arm64-efi platform.
/nix/store/q04y26jv3x166mij9nlxkwm3mzjbs3m5-grub-2.06/sbin/grub-install: error: /boot doesn't look like an EFI partition.
/nix/store/z3ldw968n707hpzp3b8p8vzkaxqilick-install-grub.pl: installation of GRUB EFI into /boot failed: No such file or directory
[ 72.900456] reboot: Power down
error: builder for '/nix/store/04ss5jq7vn7q6kigw2lqd6mxx2lwix0v-nixos-disk-image.drv' failed with exit code 1
hmm, no, as you can see it fallsback to tcg. but for some reason the efi partition seems to be missing. not sure why though
but I was able to reproduce, debugging now
alright, I created a PR https://github.com/nix-community/nixos-generators/pull/152 can you check if that fixes it for you? thanks
@Lassulus working!!!! 😄
[ 1.533052] random: fast init done
[ 2.218528] random: crng init done
[ 11.524497] reboot: Restarting system
WARNING: Image format was not specified for 'nixos.raw' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
Could not access KVM kernel module: No such file or directory
qemu-system-aarch64: failed to initialize kvm: No such file or directory
qemu-system-aarch64: falling back to tcg
loading kernel modules...
[ 9.205322] Module has invalid ELF structures
[ 9.362234] Module has invalid ELF structures
mounting Nix store...
mounting host's temporary directory...
starting stage 2 (/nix/store/m2wqdxfq14jid18h7pqn1p656i4jhr57-vm-run-stage2)
mkfs.fat 4.2 (2021-01-31)
setting up /etc...
/etc/tmpfiles.d/journal-nocow.conf:26: Failed to resolve specifier: uninitialized /etc detected, skipping
All rules containing unresolvable specifiers will be skipped.
updating GRUB 2 menu...
installing the GRUB 2 EFI boot loader into /boot...
Installing for arm64-efi platform.
Installation finished. No error reported.
tune2fs 1.46.5 (30-Dec-2021)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds
Setting time filesystem last checked to Tue Apr 26 17:45:52 2022
[ 101.290172] reboot: Power down
/nix/store/i6xpswr9n7gjsp51cha3n9afg1m8c9ww-nixos-disk-image/nixos.img
hmm, its hard to fix this generally, but I can try to come up with some mkIfs in the qcow format
I created this PR, would be nice if someone could test it: https://github.com/nix-community/nixos-generators/pull/174
should be fixed by https://github.com/nix-community/nixos-generators/pull/174