DietPi icon indicating copy to clipboard operation
DietPi copied to clipboard

NanoPi ZeroPi Images freezing

Open TimoAbt opened this issue 4 months ago • 15 comments

Creating a bug report/issue

  • [x] I have searched the existing open and closed issues

Required Information

  • Tested images:
    • DietPi_ZeroPi-ARMv7-Trixie.img.xz -> freezes
    • DietPi_ZeroPi-ARMv7-Bookworm.img.xz -> freezes
    • DietPi_ZeroPi-ARMv7-Forky.img.xz -> freezes
    • DietPi_NanoPiNEO-ARMv7-Trixie.img.xz -> works, but without Ethernet (to be expected)
    • h3-sd-friendlycore-focal-4.14-armhf-20250402.img.gz -> works, including Ethernet
  • SBC model | NanoPi ZeroPi 512M
  • Power supply used | different power supplies tested
  • SD card used | different SD cards tested

Additional Information (if applicable)

  • Was the software title installed freshly or updated/migrated? fresh
  • Can this issue be replicated on a fresh installation of DietPi? happens always

Steps to reproduce

  1. Flash image to SD card
  2. Insert into ZeroPi
  3. Turn on power
  4. After a few seconds it freezes (Heartbeat-LED and UART)

Extra details

  • Freeze seems to be random, but always relatively quick (seconds).
  • Slower SD card might take a bit longer till it freezes

If there is any way I can help please let me know.

TimoAbt avatar Sep 09 '25 16:09 TimoAbt

Can you provide any logs, at best from serial console?

MichaIng avatar Sep 09 '25 16:09 MichaIng

I hope that helps: bootlog3.txt

Edit: Better Formatting: bootlog5.txt

TimoAbt avatar Sep 09 '25 17:09 TimoAbt

Okay at first view it really seems to end randomly. Thought first there would have been an issue with partition expansion, but since the service disables itself as first step, that did not run on 2nd boot.

But it ends pretty much after

[  OK  ] Found device dev-ttyS0.device - /dev/ttyS0.

in both cases. I guess after that one, the very serial console device I would expect you are connected to. But maybe that is false assumption, and you are connected to a different UART, or naming changed, or it is somehow setup falsely by the kernel. In U-Boot:

ns16550_serial serial@1c28000

In kernel:

[    2.753236] 1c28000.serial: ttyS0 at MMIO 0x1c28000 (irq = 136, base_baud = 1500000) is a 16550A

So that at least indicates the same device. It supports 1.5M baud rate? Or is the kernel assuming/obtaining this info falsely? Wiki days 115200, and I never heard of something else for Allwinner SoCs: https://wiki.friendlyelec.com/wiki/index.php/ZeroPi#Layout

I checked Linux code changes (in related device trees) and Armbian patches but there was no recent change in this regards.

  • Does the serial connection always terminate right/shortly after Found device dev-ttyS0.device?
  • Does LED switch from heartbeat to steady light the very same moment?
  • Can you verify boot is really stack at that moment, and that not just output on the serial console stops, i.e. let it run for somewhat longer and try to connect via SSH.
  • Any change it is a powering issue, i.e. are you using a proper dedicated PSU?

MichaIng avatar Sep 09 '25 18:09 MichaIng

I also generated with recent kernel and U-Boot, which can be found here: https://dietpi.com/downloads/images/testing/

EDIT: Serial debug port on my NanoPi M1 works well with that kernel at least, using the exact same UART device of the Allwinner H3.

MichaIng avatar Sep 09 '25 18:09 MichaIng

  • Heartbeat and UART always freeze at the same time
  • I'm connected with 115200 (sudo picocom -b 115200 /dev/ttyUSB1)
  • It doesn't connect to the network (I've given it a few minutes a few times). I only tried UART after changing SD cards and power supplies did nothing. I have a NanoPi Neo 256M and a Neo 3 2GB, both work without issues.
  • I tried a 5V 2A PSU, a powerbank, the always on charging USB-Port on the Thinkpad Docking Station, no change. The NanoPi Neo 3 runs with all 3 and needs a lot more power (The NanoPi Neo 256M, which is almost the same as the NanoPi ZeroPi, runs without issues off a 5V 500mA USB-Port)
  • The testing image behaves the same

But you are right, it always seems to freeze arount the dev-ttyS0.device I tried it 4 more times (with the testing image), 2x with Samsung 128GB Evo Select, 1x SanDisk 4GB class 4, 1x NoName 8GB class 8 (so two old slow cards), it always freezes there bootlog6.txt bootlog7.txt bootlog8.txt bootlog9.txt

Just for comparison here I did it with the NanoPi Neo image (that one never freezes): bootlog10.txt

TimoAbt avatar Sep 10 '25 00:09 TimoAbt

Did you try the image from the testing directory I linked above?

MichaIng avatar Sep 10 '25 16:09 MichaIng

Yes, bootlog6 - bootlog9 in the previous comment are all with the testing directory you linked.

TimoAbt avatar Sep 10 '25 17:09 TimoAbt

Same issue with Armbian. I'm currently trying when the issue first appeared. Armbian 25.5.0 works, 25.11.0 doesn't.

TimoAbt avatar Sep 11 '25 17:09 TimoAbt

not working: Armbian_community_25.8.0-trunk.338_Zeropi_bookworm_current_6.12.35_minimal.img.xz bootlog12.txt

working: Armbian_community_25.5.0-trunk.256_Zeropi_bookworm_legacy_6.1.104_minimal.img.xz bootlog13.txt

TimoAbt avatar Sep 11 '25 18:09 TimoAbt

Does it help to mask the dev-ttyS0.device unit? I.e. create a symlink named dev-ttyS0.device to /dev/null in /etc/systemd/system before booting the device. If systemd setting up this device really causes the issue, that should fix it. It prevents the login prompt, AFAIK, but you should still see all first boot output, network setup etc on the serial console, to know when SSH is assumed to be connectable (in case masking that unit really helps).

MichaIng avatar Sep 13 '25 21:09 MichaIng

I used the image from your linked testing directory and created the symlink before boot (sudo ln -s /dev/null etc/systemd/system/dev-ttyS0.device) Now it freezes on [ OK ] Finished systemd-random-seed.service - Load/Save OS Random Seed. Tried it twice, same result. bootlog16.txt

TimoAbt avatar Sep 14 '25 07:09 TimoAbt

Okay, so the UART device is most likely unrelated, but just by chance one of the last things in the log. One last idea I have to get some more output is setting udev_log=debug in /etc/udev/udev.conf, so we see potentially last configured devices and udev RUN commands.

Since the NanoPi NEO image works, and the only difference between the two is the used device tree, I'll compare those two.

MichaIng avatar Sep 14 '25 15:09 MichaIng

Tried it with it, but I think it might freeze too early: bootlog17.txt

TimoAbt avatar Sep 14 '25 16:09 TimoAbt

Or maybe udev does not log to the serial console somehow. At that point, it should show something already. You uncommented the setting, right?

However, I checked the device trees:

  • ZeroPi: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/arm/boot/dts/allwinner/sun8i-h3-zeropi.dts?h=linux-6.12.y
  • NanoPi NEO: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/arm/boot/dts/allwinner/sun8i-h3-nanopi-neo.dts?h=linux-6.12.y

The only difference is, aside of the different Ethernet setup on the ZeroPi with dedicated PHY and regulator, is the micro USB port:

  • On the NanoPi NEO it is attached to a USB controller, hence can be theoretically used in OTG or peripheral (default) mode, and potentially in host mode, if the board is powered otherwise. So the USB PHY GPIOs for that are defined as well. Aside of the USB-A port, theoretically, the left 2 USB controllers can be accessed via GPIO as well, but those are disabled in the mainline device tree. We do however enable them via device tree overlays in our image. This all looks consistent with the schematics: https://wiki.friendlyelec.com/wiki/images/f/fd/Schematic_NanoPi-NEO-V1.4-1801-20180320.pdf
  • On the ZeroPi that port is not attached to any USB controller, hence that controller is also not enabled. What is weird is that the port's mode is explicitly set to "host". Not sure whether the default ("otg") causes issues if there is no controller attached, but obviously "peripheral" does not cause serious issues, since the NEO image works. The schematics show that only the USB3 controller (the USB-A port) is physically attached, while micro USB port should not be able to do anything else than powering the device: https://wiki.friendlyelec.com/wiki/images/7/71/ZeroPi_20190731_Schematic.pdf This also matches the info on the wiki "for power input only": https://wiki.friendlyelec.com/wiki/index.php/ZeroPi#Hardware_Spec

Two 3 things we can test:

  1. Remove usbhost1 usbhost2 from the overlays= line in /boot/dietpiEnv.txt. While with the NEO image, this does not seem to cause issues, it is at least nonsense since the ZeroPi does not make use of those 2 controllers via GPIO ports, like the NEO does. Or am I missing something and there are revisions with additional GPIO header or so? I just wonder why we enabled those overlays in the first place for the ZeroPi image explicitly.
  2. Switch the micro USB port to "peripheral" mode, just like it is on the NanoPi NEO image. This could be done with a device tree overlay generated on another ARM board, or the same with a functional image. For simplicity, you could just boot the same image with the NEO device tree, by adding
    fdtfile=sun8i-h3-nanopi-neo.dtb
    
    to /boot/dietpiEnv.txt. Then these commands to add the overlay:
    sudo apt install device-tree-compiler # ah shit, this will be difficult without functional Ethernet :(
    sudo mkdir -p /boot/overlay-user
    cat << '_EOF_' | sudo tee /boot/overlay-user/usb-mode.dtso > /dev/null
    /dts-v1/;
    /plugin/;
    / {
    	fragment@0 {
    		target = <&usb_otg>;
    		__overlay__ {
    			dr_mode = "peripheral";
    		};
    	};
    };
    _EOF_
    sudo dtc -I dts -O dtb -o /boot/overlay-user/usb-mode.dtbo /boot/overlay-user/usb-mode.dtso
    G_SUDO G_CONFIG_INJECT 'user_overlays=' 'user_overlays=usb-mode' /boot/dietpiEnv.txt
    
    Installing device-tree-compiler of course requires Internet access, maybe you have a WiFi adapter, otherwise the dtbo can be created on another ARM system as well. Or I can do a kernel built with this mode patched or the overlay added.
  3. Enabling the USB0 controller like it is done with the NEO image. Though it would be really weird if that made any difference, since that controller is not wired to anything according to schematics. This however is easy again with an existing overlay, setting this in /boot/dietpiEnv.txt:
    overlays=usbhost0
    

If none of this makes any difference, the only left thing is the Ethernet definitions.

Ah, one more: https://github.com/armbian/build/blob/main/patch/kernel/archive/sunxi-6.12/patches.armbian/arm-dts-h3-nanopi-neo-Add-regulator-leds-mmc2.patch Armbian patches some regulators and an additional MMC node inside. Sadly without any further explanation, and I am too lazy to track down the commit. It mentions "eMMC", which is weird, since that board has no eMMC flash or socket, and the SoC does not natively support it either, AFAIK. Only thing I could think of is that it is somehow possible to attach an eMMC device via 12 pins header, maybe the USB related pins. The CPU regulator could theoretically change something about the board's stability, so we could add this patch to the ZeroPi device tree our end for testing.

MichaIng avatar Sep 14 '25 16:09 MichaIng

Yeah, so enabling the USB1 and USB2 controller overlays on ZeroPi is nonsense, hence removing those in any case: https://github.com/MichaIng/DietPi/commit/34bb9d9

When they were added to the kernel, they were enabled for some H3 boards, which really have pins wired to them: https://github.com/armbian/build/commit/e1ab0cb

It was then blindly adapted for the ZeroPi, even that it does not have any pins aside of the debug UART: https://github.com/armbian/build/commit/bea1df9

And we did blindly adapt the Armbian defaults, without checking whether it makes sense in this case.

MichaIng avatar Sep 14 '25 17:09 MichaIng