VisionFive 2 NVME boot

[RossComputerGuy]

Saw that in #964, NVME boot is possible. Trying it and it's not working:

<<< NixOS Stage 1 >>>

loading module clk-starfive-jh7110-aon...
[    3.243664] gpio gpiochip1: Static allocation of GPIO base is deprecated, use dynamic allocation.
[    3.247057] mmc_host mmc1: Bus speed (slot 0) = 204799999Hz (slot req 300000Hz, actual 299415HZ div = 342)
[    3.252929] starfive-jh7110-aon-pinctrl 17020000.pinctrl: StarFive GPIO chip registered 4 GPIOs
loading module clk-starfive-jh7110-stg...
loading module phy-jh7110-pcie...
loading module p[    3.291095] mmc_host mmc0: Bus speed (slot 0) = 204799999Hz (slot req 300000Hz, actual 299415HZ div = 342)
[    3.299676] pcie-starfive 2b000000.pcie: error -ENODEV: failed to get valid pcie domain

[    3.299843] pcie-starfive 2c000000.pcie: error -ENODEV: failed to get valid pcie domain
loading module nvme...
loading module dm_mod...
[    3.542940] device-mapper: ioctl: 4.48.0-ioctl (2023-03-01) initialised: [email protected]
running udev...
[    3.609137] mmc_host mmc1: Bus speed (slot 0) = 204799999Hz (slot req 50000000Hz, actual 34133333HZ div = 3)
Starting systemd-udevd version 2[    3.620683] mmc1: new high speed SDXC card at address aaaa
56.8
kbd_mode: KDSKBMODE: Inappropriate ioctl for device
Gstarting device mapper and LVM...
File descriptor 8 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
File descriptor 9 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
waiting for device /dev/disk/by-uuid/997fc0c7-2860-4924-a732-04493c579f20 to appear...File descriptor 3 (/nix/store/r6hn0pf2lvxxcjkg2vsncdv3wk75zpim-initrd-fsinfo) leaked on l
vm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
File descriptor 8 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
File descriptor 9 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
.File descriptor 3 (/nix/store/r6hn0pf2lvxxcjkg2vsncdv3wk75zpim-initrd-fsinfo) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/
bin/ash
File descriptor 8 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
File descriptor 9 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
.File descriptor 3 (/nix/store/r6hn0pf2lvxxcjkg2vsncdv3wk75zpim-initrd-fsinfo) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/
bin/ash
File descriptor 8 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
File descriptor 9 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
.File descriptor 3 (/nix/store/r6hn0pf2lvxxcjkg2vsncdv3wk75zpim-initrd-fsinfo) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/
bin/ash
File descriptor 8 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
File descriptor 9 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
.File descriptor 3 (/nix/store/r6hn0pf2lvxxcjkg2vsncdv3wk75zpim-initrd-fsinfo) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/
bin/ash
File descriptor 8 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
File descriptor 9 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
.File descriptor 3 (/nix/store/r6hn0pf2lvxxcjkg2vsncdv3wk75zpim-initrd-fsinfo) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/
bin/ash
File descriptor 8 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
File descriptor 9 (/dev/console) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/bin/ash
.File descriptor 3 (/nix/store/r6hn0pf2lvxxcjkg2vsncdv3wk75zpim-initrd-fsinfo) leaked on lvm invocation. Parent PID 1: /nix/store/4nc7aajxlpclfb1gx0hp5d1kswdjprb3-extra-utils/
bin/ash
Timed out waiting for device /dev/disk/by-uuid/997fc0c7-2860-4924-a732-04493c579f20, trying to mount anyway.
mounting /dev/disk/by-uuid/997fc0c7-2860-4924-a732-04493c579f20 on /...
[   43.434639] /dev/disk/by-uuid/997fc0c7-2860-4924-a732-04493c579f20: Can't lookup blockdev
mount: mounting /dev/disk/by-uuid/997fc0c7-2860-4924-a732-04493c579f20 on /mnt-root/ failed: No such file or directory

Meanwhile the installer image can see the NVME on boot:

[    6.291783] pcie-starfive 9c0000000.pcie: port link up
[    6.297978] pcie-starfive 9c0000000.pcie: PCI host bridge to bus 0001:00
[    6.299790] usb 1-1: new high-speed USB device number 2 using xhci_hcd
[    6.304747] pci_bus 0001:00: root bus resource [bus 00-ff]
[    6.317018] pci_bus 0001:00: root bus resource [mem 0x38000000-0x3fffffff]
[    6.323933] pci_bus 0001:00: root bus resource [mem 0x980000000-0x9bfffffff pref]
[    6.331556] pci 0001:00:00.0: [1556:1111] type 01 class 0x060400 PCIe Root Port
[    6.338923] pci 0001:00:00.0: PCI bridge to [bus 01]
[    6.343912] pci 0001:00:00.0:   bridge window [mem 0x38000000-0x380fffff]
[    6.350722] pci 0001:00:00.0:   bridge window [mem 0x900000000-0x9000fffff 64bit pref]
[    6.359042] pci 0001:00:00.0: supports D1 D2
[    6.363343] pci 0001:00:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    6.371530] pci 0001:01:00.0: [144d:a808] type 00 class 0x010802 PCIe Endpoint
[    6.378797] pci 0001:01:00.0: BAR 0 [mem 0x00000000-0x00003fff 64bit]
[    6.385433] pci 0001:01:00.0: 4.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x1 link at 0001:00:00.0 (capable of 31.504 Gb/s with 8.0 GT/s PCIe x4 link)
[    6.407795] pci_bus 0001:01: busn_res: [bus 01-ff] end is updated to 01
[    6.414474] pci 0001:00:00.0: bridge window [mem 0x38000000-0x380fffff]: assigned
[    6.421982] pci 0001:01:00.0: BAR 0 [mem 0x38000000-0x38003fff 64bit]: assigned
[    6.429337] pci 0001:00:00.0: PCI bridge to [bus 01]
[    6.434324] pci 0001:00:00.0:   bridge window [mem 0x38000000-0x380fffff]
[    6.441150] pci_bus 0001:00: resource 4 [mem 0x38000000-0x3fffffff]
[    6.447433] hub 1-1:1.0: USB hub found
[    6.447441] pci_bus 0001:00: resource 5 [mem 0x980000000-0x9bfffffff pref]
[    6.451340] hub 1-1:1.0: 4 ports detected
[    6.458076] pci_bus 0001:01: resource 1 [mem 0x38000000-0x380fffff]
[    6.458411] pcieport 0001:00:00.0: PME: Signaling with IRQ 56
loading module nvme...
[    6.573825] nvme nvme0: pci function 0001:01:00.0
[    6.578599] nvme 0001:01:00.0: enabling device (0000 -> 0002)
[    6.587231] nvme nvme0: missing or invalid SUBNQN field.
[    6.592879] nvme nvme0: D3 entry latency set to 8 seconds
[    6.599815] mmc_host mmc0: Bus speed (slot 0) = 45511111Hz (slot req 100000Hz, actual 99805HZ div = 228)
[    6.605750] hwmon hwmon0: temp1_input not attached to any thermal zone
[    6.615884] hwmon hwmon0: temp2_input not attached to any thermal zone
[    6.622441] hwmon hwmon0: temp3_input not attached to any thermal zone
[    6.639329] nvme nvme0: 4/0/0 default/read/poll queues
[    6.655998]  nvme0n1: p1 p2
loading module dm_mod...
[    6.732731] device-mapper: ioctl: 4.48.0-ioctl (2023-03-01) initialised: [email protected]

Device config: https://github.com/RossComputerGuy/dotfiles/blob/master/devices/jegan/default.nix

CC @onny since they got it working before.