darp6: long boot time after certain sequences of boots/power-offs

[jacobgkau]

On the darp6, a normal boot (from pushing the power button to seeing the decryption prompt) takes ~8 seconds. The first time booting after flashing firmware, it's normal to see the boot take ~28 seconds instead, where the extra delay happens before the System76 logo/splash screen appears.

On the current stable firmware, I'm able to make the boot take ~28 seconds again by performing the following testing procedure (there may be a faster way to make this occur, but this is a reliable method I know of right now):

• Plug in an NVMe enclosure to the back-left USB port, boot while holding down Esc, enter the One Time Boot menu to confirm the NVMe enclosure is showing up, then push the power button to turn the machine off (without finishing the boot.)
• Move the NVMe drive to the front-left USB port, repeat.
• Move the NVMe drive to the right USB port, repeat.
• Test the three ports a second time in the same order.
• Repeat the above process with a System76 flash drive, then with a generic flash drive.
• Boot to the decryption screen, then power off the machine (before decrypting.) Repeat this step until a long boot occurs (should happen on the second or third boot.)

On current stable, the long boot only happens once, then subsequent boots go back down to the normal time. On the master branch of firmware-open, once the long boot occurs, subsequent boots also take the longer amount of time, making this a more noticeable issue. I've tried reseating/changing out the RAM and SSD, resetting the CMOS, and clearing entries from NVRAM using efibootmgr, and I haven't found a way to stop the long boot time once this occurs short of re-flashing the firmware.

[crawfxrd]

We should document the first boot after flashing takes longer due to memory training.

Can you get it to happen and provide the output of cbmem?

cd coreboot/util/cbmem
make
sudo ./cbmem

[jacobgkau]

Here is cbmem on a long boot (I have it in the state where it's doing it every time.) darp6-longboot-cbmem.txt

Looks like the gap is at 950:calling FspMemoryInit.

[jacobgkau]

I tried a warm reboot (instead of power off/power on) and it did not seem to hang this time, here's cbmem from that: darp6-reboot-cbmem.txt

I then powered off, waited a few seconds, and powered back on, the hang occurred again and here is cbmem: darp6-coldboot-2.txt

[jacobgkau]

Since flashing Open EC onto the darp6 earlier today, I have not seen any more long boots (aside from the first boot after a flash.) The sequence that was reliably recreating the issue before is no longer recreating the issue.

[leviport]

Galp4 doesn't seem to be having this same issue. Since it only has 1 USB port on the left, I tried both booting twice from the same port, and just booting once on either USB port.

[jackpot51]

@jacobgkau it's pretty clearly running memory initialization again. I'm hoping we can solve this with the transition update