raspberry-pi-pcie-devices icon indicating copy to clipboard operation
raspberry-pi-pcie-devices copied to clipboard

Test Pine64 SOQuartz on CM4 boards

Open geerlingguy opened this issue 3 years ago • 243 comments

As with the Radxa CM3, I would also like to test Pine64's SOQuartz with some CM4 boards, since it's supposed to be pin-compatible.

DSC04914

DSC04917

@timonsku mentioned the Wiki (linked above) and this dtb artifact are the two best ways to get started with it. I'd like to write up my experience trying to get the thing to boot, and also seeing if it fits and works in a few popular CM4 boards (starting with the official IO Board).

geerlingguy avatar Dec 05 '21 18:12 geerlingguy

IMG_0107

So I can get this thing to boot... at least somewhat:

  1. I flashed this dtb image to a microSD card using Balena Etcher.
  2. I popped that microSD card and the SOQuartz onto a CM4 IO Board.
  3. I plugged in the board to my Mac via UART.
  4. I plugged my HDMI display into HDMI0.
  5. I plugged in power to the board.

Using screen /dev/tty.usbserial-0001 115200 I got gibberish. Also tried 1500000 (which seemed to be a default for the RK3566?), and 9600, and still got gibberish. But it was getting data, which means something's flowing through.

On the HDMI display, after a few seconds, I see a flashing cursor. So that's impressive, I guess :D

IMG_0106

geerlingguy avatar Dec 09 '21 19:12 geerlingguy

Also tested UART with a FT232R adapter, but that got nothing at all (not even gibberish) at 1.5 Mbps; see this comment: https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/327#issuecomment-990317145

geerlingguy avatar Dec 09 '21 22:12 geerlingguy

If you disassemble the used installation image of the SD card into parts and extract the DTB file from it.

dtc -I dtb -O dts rk3566-soquartz-cm4.dtb> rk3566-soquartz-cm4.dts

Convert it to DTS filethen you can see such lines in it:

chosen { stdout-path = "serial2: 1500000n8"; };

This means that serial line 2 is used (from the point of view of RADXA CM3) at a speed of 1500000. If you do not see normal output, there are two possibilities.

  1. You connected the cable not to serial2 but to another serial and therefore you see garbage. The on-board console is the RPI CM4 console. And you need to use the console from the RADXA CM3 point of view.

  2. Your console cable is simply stray (too long) or the FT232R chip is not original.

The original FT232R cable with Radxa QURTZ64A works for me.

I usually use cables based on the CH304B chip, but they have even more fakes. You are doing everything right, you just need to find the error.

Good luck

S199pWa1k9r avatar Dec 10 '21 17:12 S199pWa1k9r

@S199pWa1k9r - Thanks for the pointers! For now, I have to finish up a couple other things before the weekend, so I'll just hold off and hope the CH340 adapter is genuine and works out of the box (might also try with CoolTerm).

geerlingguy avatar Dec 10 '21 18:12 geerlingguy

Maybe you need to use the GPIO14-TX, GPIO15-RX pins to connect to the UART console?

It is written here https://files.pine64.org/doc/quartz64/SOQuartz%20Connector%20Pin%20Assignments%20ver%201.0.ods

S199pWa1k9r avatar Dec 10 '21 19:12 S199pWa1k9r

Aha! At @hipboi's suggestion, I installed CoolTerm—it works fine with my existing USB-UART adapter at 1.5 Mbps, and now I can see the SOQuartz's output just fine.

I noticed during boot it paused for about 5 seconds on this prompt, then auto-selected #1 ("Buildroot-recovery"):

FIT: No FIT image
Could not find misc partition
ANDROID: reboot reason: "(none)"
optee api revision: 2.0
TEEC: Waring: Could not find security partition
Not AVB images, AVB skip
No valid android hdr
Android image load failed
Android boot failed, error -1.
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:5...
Found /extlinux/extlinux.conf
Retrieving file: /extlinux/extlinux.conf
reading /extlinux/extlinux.conf
812 bytes read in 4 ms (198.2 KiB/s)
Quartz64 Installer
1:.Buildroot-recovery
2:.Debian-Installer
3:.Boot Root SDMMC
4:.Boot Root eMMC
Enter choice: 1:.Buildroot-recovery

So I rebooted, then made sure to press 2, then Enter to get into the Debian installer.

heh... and now the data coming back expects a full terminal with colors and such:

[47m .[31m[!!] Select a language.[30m .(0tqqqqqqqqqqqqqqqqqqqqqqqqk.[6;3Hx.[0m.(B.[30m.[47m                                                                         .(0x.[0m.(B.[1m.[37m.[40m .[7;3H.[0m.(0.[30m.[47mx.[0m.(B.[30m.[47m Choose the languagsed for the installation process. The        .(0x.[0m.(B.[1m.[3740m .[8;3H.[0m.(0.[30m.[47mx.[0m.(B.[30m.[47m selected language lso be the default language for the installed   .(0x.[0m.(B.[1m.7m.[40m .[9;3H.[0m.(0.[30m.[47mx.[0m.(B.[30m.[47m system.                                                                 .(0x.[0m.(B.[1m.[37m.[40m .[10;3H.[0m.(0.[30m.[47mx.[0m.(B.[30m.[47m                                                                  .(0x.[0m.(B.[1m.[37m.[40m .[11;3H.[0m.(0.[30m.[47mx.[0m.(B.[30m.anguage:                                                            .(0x.[0m.(B.[1m.[37m.[40m .[12;3H.[0m.(0.[30m.[47mx.[0m.(B.[30m.[47m                                                                      .(0x.[0m.(B.[1m.[37m.[40m .[13;3H.[0m.(0.[30m.[47mx.[0m.(B.[30m.[47m                               C                                         .(0x.[0m.(B.[1m.[37m.[40m .[14;3H.[0m.(0.[30m.[47mx.[0m.(B.[30m.[47m                               .[7m.[41mEnglish.[30m.[47m                                   .(0x.m.(B.[1m.[37m.[40m .[15;3H.[0m.(0.[30m.[47mx.[0m.(B.[30m.[47m                                                                         .(0x.[0m.(B.[1m.[37m.[40m .[16;3H.[0m.(0.[30m.[47mx.[0m.(B.[30m.[47m     <Go Back>                                                          .(0x.[0m.(B.[1m.[37m.[40m .[17;3H.[0m.(0.[30m.                                         .(0x.[0m.(B.[1m.[37m.[4qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj.[0m.(B.[1m.                                              
.[5B.[44m<Tab> mo.[14;35H.[H.[0m.[7m.[m.[32m.[40m[           .[37m.[31m (.[37m1*installer.[31m).[37m  2 shell  3 shell  4- log           .[32m][.[34m Jan 01 .[37m 0:00 .[32m].[14;35H.[0m

geerlingguy avatar Dec 15 '21 18:12 geerlingguy

Working through the installer with this formatting is ... fun. But I seem to have gotten most of the way through so far. This was a funny little tidbit:

Screen Shot 2021-12-15 at 12 33 56 PM

iu

Though I'm hitting an error deeper in:

No kernel modules were found. This probably is due to a mismatch between the kernel used by this version of the installer and the kernel version available in the archive.

Going to put this on hold until I can figure out a way to interact via TTY that doesn't make my brain hurt in serial console... is there a way?

geerlingguy avatar Dec 15 '21 18:12 geerlingguy

I use Putty for this, gave me color menus and everything

timonsku avatar Dec 15 '21 18:12 timonsku

Curious, since this is such a non-standard baud rate... could using the "Arduino as a TTL" trick still work here?

miguemely avatar Dec 15 '21 18:12 miguemely

I went through the installation btw. it did not yield a working system. Bootloader is missing and need to manually call the init system, have not figured that out yet

timonsku avatar Dec 15 '21 18:12 timonsku

@miguemely - Yeah, or a Pico or Pi probably, but as it turns out CoolTerm supports 1.5 Mbps with the cheap USB UART adapter I've been using, so at this point it's a matter of CoolTerm settings so my brain doesn't hurt through the interactive installer :)

geerlingguy avatar Dec 15 '21 19:12 geerlingguy

Perfect... Time to pull the 3 SOQuartz I have out of the shelf now.

miguemely avatar Dec 15 '21 19:12 miguemely

@geerlingguy Are you using GPIO15-16, or 14-15?

Edit: Using GPIO15-16, 15 connected to TX, 16 connected to RX of the Arduino, and reset held:

image

Don't I love some gibberish.

miguemely avatar Dec 15 '21 20:12 miguemely

@geerlingguy

here is mine test with the image you mentioned above

image

Using putty on Windows 11 with FT232B USB to Serial dongle. GPIO14 as TX and GPIO15 as RX IMG_9962

and if you select the '2:.Debian-Installer' it would start the debian installer like this image

martinx72 avatar Dec 15 '21 23:12 martinx72

Alright, Arduino = Bad idea here.

https://www.amazon.com/gp/product/B08TX3KTP1/ worked natively on Windows. Same with https://www.amazon.com/gp/product/B00LZV1G6K/

image

Edit: I tried Manjaro's build but it seems stuck somewhere. I let it idle and came back to a call trace.

Retrieving file: /initramfs-linux.img
reading /initramfs-linux.img
8419052 bytes read in 699 ms (11.5 MiB/s)
Retrieving file: /Image
reading /Image
30962176 bytes read in 2559 ms (11.5 MiB/s)
append: initrd=/initramfs-linux.img root=PARTUUID=09a4ee0c-c5a3-42ca-8084-fa9c7daec648 rw earlycon=uart8250,mmio32,0xfe660000 console=tty1 console=ttyS2,1500000n8 quiet splash plymouth.ignore-serial-consoles
Retrieving file: /dtbs/rockchip/rk3566-quartz64-a.dtb
reading /dtbs/rockchip/rk3566-quartz64-a.dtb
101338 bytes read in 17 ms (5.7 MiB/s)
Fdt Ramdisk skip relocation
## Flattened Device Tree blob at 0x0a100000
   Booting using the fdt blob at 0x0a100000
   Using Device Tree in place at 000000000a100000, end 000000000a11bbd9
can't found rockchip,drm-logo, use rockchip,fb-logo
WARNING: could not set reg FDT_ERR_BADOFFSET.
failed to reserve fb-loader-logo memory
Adding bank: 0x00200000 - 0x08400000 (size: 0x08200000)
Adding bank: 0x09400000 - 0xf0000000 (size: 0xe6c00000)
Adding bank: 0x1f0000000 - 0x200000000 (size: 0x10000000)
Total: 3676.925 ms

Starting kernel ...

[    0.301092] arm-scmi firmware:scmi: Failed. SCMI protocol 22 not active.
[   60.392529] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[   60.393098] rcu:     1-...0: (0 ticks this GP) idle=4c7/1/0x4000000000000000 softirq=17/17 fqs=3001
[   60.393904]  (detected by 3, t=6002 jiffies, g=-1147, q=185)
[ 1928.221803] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 1928.221824] watchdog: BUG: soft lockup - CPU#3 stuck for 1635s! [swapper/3:0]
[ 1928.222385] rcu:     1-...0: (0 ticks this GP) idle=4c7/1/0x4000000000000000 softirq=17/17 fqs=8776
[ 1928.223019] Modules linked in:
[ 1928.223804]  (detected by 2, t=192784 jiffies, g=-1147, q=187)
[ 1928.224088] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 5.16.0-rc4-5-MANJARO-ARM #1
[ 1928.225264] Hardware name: Pine64 RK3566 Quartz64-A Board (DT)
[ 1928.225789] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 1928.225792]  stack:    0 pid:    1 ppid:     0 flags:0x0000000a
[ 1928.225802] pc : arch_cpu_idle+0x18/0x2c
[ 1928.226943] lr : arch_cpu_idle+0x14/0x2c
[ 1928.227643] sp : ffff800011fbbde0
[ 1928.227943] x29: ffff800011fbbde0 x28: 0000000000000000 x27: 0000000000000000
[ 1928.228595] x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000000
[ 1928.229244] x23: 0000000000000000 x22: ffff0001f0181e00 x21: 0000000000000000
[ 1928.229894] x20: ffff0001f0181e00 x19: 0000000000000000 x18: 0000000000000020
[ 1928.230542] x17: 00000000c3c0b9c4 x16: 000000002ef522c7 x15: ffff0001f026a388
[ 1928.231190] x14: 000000000000018b x13: 0000000000000001 x12: 0000000000000000
[ 1928.231841] x11: 0000000000000000 x10: 0000000000000ab0 x9 : ffff800011fbbd80
[ 1928.232491] x8 : ffff0001f0182910 x7 : ffff8000117f2080 x6 : 00000000fbfa2774
[ 1928.233139] x5 : 00000000000000c0 x4 : 000000000000896e x3 : ffff8001edf6f000
[ 1928.233788] x2 : 0000000000011585 x1 : ffff0001ff761f28 x0 : 00000000000000e0
[ 1928.234439] Call trace:
[ 1928.234666]  arch_cpu_idle+0x18/0x2c
[ 1928.234996]  default_idle_call+0x24/0x6c
[ 1928.235359]  cpuidle_idle_call+0x158/0x1ac
[ 1928.235744]  do_idle+0xa4/0xf4
[ 1928.236030]  cpu_startup_entry+0x24/0x80
[ 1928.236394]  secondary_start_kernel+0xe4/0x110
[ 1928.236805]  __secondary_switched+0x94/0x98```

miguemely avatar Dec 16 '21 17:12 miguemely

If you disassemble the used installation image of the SD card into parts and extract the DTB file from it.

dtc -I dtb -O dts rk3566-soquartz-cm4.dtb> rk3566-soquartz-cm4.dts

Convert it to DTS filethen you can see such lines in it:

chosen { stdout-path = "serial2: 1500000n8"; };

so, can the .dts be edited, compiled, and written into the image to set a more conventional speed?

(not that I can plug my terminal into one of these without frying them)

JustCommitRandomness avatar Dec 17 '21 16:12 JustCommitRandomness

Ok, after full install and telling to to boot to SDMMC: https://www.toptal.com/developers/hastebin/uyibicurod.yaml

miguemely avatar Dec 25 '21 17:12 miguemely

Any updates on getting something like a debian os with dropbear preinstalled so that the rest of us without a uart to usb can do this too?

Also how is the PCI-e support? Anybody tested? SSD's?

adminy avatar Jan 24 '22 01:01 adminy

Rockchip devices default to 1.5m baud because they cannot generate a clean standard baud below that, due to the clock source and limitations of the divider. @miguemely You were running a quartz64-a image on the SoQuartz which will damage the board.

pgwipeout avatar Feb 09 '22 22:02 pgwipeout

You were running a quartz64-a image on the SoQuartz which will damage the board.

Really? Can you explain more? @pgwipeout

hipboi avatar Feb 10 '22 01:02 hipboi

The Quartz64 and SoQuartz have different PMICs, different regulator layouts, and different power domains. Loading an incorrect DTS is never a good idea, but specifically with the SoQuartz loading the Quartz64-A DTS leads to a situation where hardware support is half broken and the board gets very hot very quickly.

pgwipeout avatar Feb 10 '22 12:02 pgwipeout

Gotcha, ok. Yeah, that could be a problem....

That's odd, I swear I saw the Manjaro build mentioned somewhere in the wiki for the SOQuartz, which is why I tried it...

miguemely avatar Feb 10 '22 13:02 miguemely

It should have a SoQuartz DTB in the image, it's just set up to load the Quartz64-A because that's what the image is built for.

pgwipeout avatar Feb 10 '22 13:02 pgwipeout

Got it, I'll probably need to re-flash the SD card, grab the SoQuartz and give it another go.

miguemely avatar Feb 10 '22 13:02 miguemely

Also, the SoQuartz dtb is specifically for a CM4 carrier module, there shouldn't be much difference with other carrier modules, but the CM4 is the only one I have to test against.

pgwipeout avatar Feb 10 '22 13:02 pgwipeout

That's the only carrier module I have as well. I just put this on the back-burner since I finally was able to get a hold of two CM4s, but have 3-4 of the SoQuartz that I would love to start tinkering with again.

miguemely avatar Feb 10 '22 13:02 miguemely

You can reach out to me on any of the Pine64 hosted services in the Quartz64 room if you have any questions.

pgwipeout avatar Feb 10 '22 13:02 pgwipeout

Alright. So, with that information, and some digging since I'm very new at this, when you flash the manjaro image, you need to modify the extlinux.conf file to use the correct DTB, since its using the quartz64-a DTB by default. It should read something like

-snip-
fdt /dtbs/rockchip/rk3566-soquartz-cm4.dtb
-snip-

Once done, take the SD card out, plop it into a CM4 carrier board, boot it up (if your using putty, make sure you turn off Flow Control), and login as root. It should drop you into the install. Follow the prompts, and you should have a working Manjaro.

Thanks @pgwipeout !

Edit: It seems the latest Manjaro minimal (Manjaro-ARM-minimal-quartz64-bsp-20220207.img.xz) does have the kernel module for the built-in wireless (AzureWave B5), however its erroring out somewhere

[    8.623505] brcmfmac: F1 signature read @0x18000000=0x15264345
[    8.638136] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[    8.640278] brcmfmac mmc2:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.pine64,soquartz-cm4.bin failed with error -2
[    8.641498] brcmfmac mmc2:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.bin failed with error -2
[    8.643057] usbcore: registered new interface driver brcmfmac
[    9.661639] brcmfmac: brcmf_sdio_htclk: HT Avail timeout (1000000): clkctl 0x50

miguemely avatar Feb 10 '22 14:02 miguemely

Yes, it doesn't include the firmware for the brcmfmac43455-sdio, you'll also need a .txt configuration.

pgwipeout avatar Feb 10 '22 16:02 pgwipeout

Has anyone managed to make PCIe work for the soquartz? I'm testing a board I designed and everything works fine except for the PCIe, which isn't detecting the attached card.

dferrg avatar Feb 22 '22 19:02 dferrg