ASM1166 SATA controller fails, SATA link down, AHCI controller unavailable, failed to resume link

[albertz]

trafficstars

Describe the bug

I have problems to use my ASM1166-based SATA controller. It is connected via M.2 to RP PCI (but I have tried other variations with ASM1166-based SATA controllers and always got the same problem).

lspci shows it as SATA controller: ASMedia Technology Inc. ASM1166 Serial ATA Controller (rev 02)

Once connected, you see some errors in dmesg (see below) and boot time is quite long (several minutes), as it seems to wait for the SATA controller. And the SATA controller does not work. The system otherwise seems to be fine.

This is also reported in other places (with some variations of the errors or symptoms):

• Raspberry Pi forum: Raspberry Pi 5 PCIe Bus Error - ASM1166
• https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/600

When editing the /boot/firmware/bcm2712-rpi-5-b.dtb file, i.e. decompiling it, then inside pcie@110000, change msi-parent to the same value as phandle, that fixes the problem. That is described here: https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/600

I don't really understand what it means to edit the dtb file, but as this editing seems to fix it, I think this should be fixed better in the kernel and/or firmware?

Steps to reproduce the behaviour

Attach an ASM1166-based SATA controller. Boot.

Device (s)

Raspberry Pi 5

System

• Machine model: Raspberry Pi 5 Model B Rev 1.0
• OS: Ubuntu 24.04 LTS
• vcgencmd version: version 30cc5f37
• uname -a: Linux serverpi5 6.8.0-1005-raspi #5-Ubuntu SMP PREEMPT_DYNAMIC Wed May 22 15:01:35 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

Logs

dmesg, when the SATA drive is attached:

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x414fd0b1]
[    0.000000] Linux version 6.8.0-1004-raspi (buildd@bos03-arm64-035) (aarch64-linux-gnu-gcc-13 (Ubuntu 13.2.0-23ubuntu4) 13.2.0, GNU ld (GNU Binutils for Ubuntu) 2.42) #4-Ubuntu SMP PREEMPT_DYNAMIC Sat Apr 20 02:29:55 UTC 2024 (Ubuntu 6.8.0-1004.4-raspi 6.8.1)
...
[    0.000000] Kernel command line: reboot=w coherent_pool=1M 8250.nr_uarts=1 pci=pcie_bus_safe snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1  smsc95xx.macaddr=2C:CF:67:1D:E3:DA vc_mem.mem_base=0x3fc00000 vc_mem.mem_size=0x40000000  console=ttyAMA10,115200 multipath=off dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc cfg80211.ieee80211_regdom=GB
...
[    2.337222] ahci 0000:01:00.0: version 3.0
[    2.372148] ahci 0000:01:00.0: enabling device (0000 -> 0002)
[    2.377943] ahci 0000:01:00.0: ASM1166 has only six ports
[    2.383366] ahci 0000:01:00.0: forcing port_map 0x33f -> 0x3f
[    2.389303] ahci 0000:01:00.0: SSS flag set, parallel bus scan disabled
[    2.396008] ahci 0000:01:00.0: AHCI 0001.0301 32 slots 32 ports 6 Gbps 0x3f impl SATA mode
[    2.404315] ahci 0000:01:00.0: flags: 64bit ncq sntf stag pm led only pio sxs deso sadm sds apst 
[    2.435601] scsi host0: ahci
[    2.438732] scsi host1: ahci
[    2.441805] scsi host2: ahci
[    2.444860] scsi host3: ahci
[    2.447907] scsi host4: ahci
[    2.450959] scsi host5: ahci
...
[    2.525188] scsi host29: ahci
[    2.528296] scsi host30: ahci
[    2.531408] scsi host31: ahci
[    2.534445] ata1: SATA max UDMA/133 abar m8192@0x1b00082000 port 0x1b00082100 irq 43 lpm-pol 0
[    2.543096] ata2: SATA max UDMA/133 abar m8192@0x1b00082000 port 0x1b00082180 irq 43 lpm-pol 0
[    2.551745] ata3: SATA max UDMA/133 abar m8192@0x1b00082000 port 0x1b00082200 irq 43 lpm-pol 0
[    2.560399] ata4: SATA max UDMA/133 abar m8192@0x1b00082000 port 0x1b00082280 irq 43 lpm-pol 0
[    2.569049] ata5: SATA max UDMA/133 abar m8192@0x1b00082000 port 0x1b00082300 irq 43 lpm-pol 0
[    2.577703] ata6: SATA max UDMA/133 abar m8192@0x1b00082000 port 0x1b00082380 irq 43 lpm-pol 0
[    2.586354] ata7: DUMMY
[    2.588804] ata8: DUMMY
[    2.591254] ata9: DUMMY
[    2.593702] ata10: DUMMY
...
[    6.186644] ata1: SATA link down (SStatus FFFFFFFF SControl FFFFFFFF)
[    6.450379] ahci 0000:01:00.0: AHCI controller unavailable!
[    6.745463] ahci 0000:01:00.0: AHCI controller unavailable!
[   13.851180] ata2: failed to resume link (SControl FFFFFFFF)
[   18.486511] ata2: SATA link down (SStatus FFFFFFFF SControl FFFFFFFF)
[   20.229121] ahci 0000:01:00.0: AHCI controller unavailable!
[   20.813486] ahci 0000:01:00.0: AHCI controller unavailable!
[   28.787205] ata3: failed to resume link (SControl FFFFFFFF)
[   33.422507] ata3: SATA link down (SStatus FFFFFFFF SControl FFFFFFFF)
[   35.165114] ahci 0000:01:00.0: AHCI controller unavailable!
[   35.749467] ahci 0000:01:00.0: AHCI controller unavailable!
[   43.723197] ata4: failed to resume link (SControl FFFFFFFF)
[   48.358487] ata4: SATA link down (SStatus FFFFFFFF SControl FFFFFFFF)
[   50.101096] ahci 0000:01:00.0: AHCI controller unavailable!
[   50.685459] ahci 0000:01:00.0: AHCI controller unavailable!
[   58.659196] ata5: failed to resume link (SControl FFFFFFFF)
[   63.294474] ata5: SATA link down (SStatus FFFFFFFF SControl FFFFFFFF)
[   65.037082] ahci 0000:01:00.0: AHCI controller unavailable!
[   65.621433] ahci 0000:01:00.0: AHCI controller unavailable!
[   73.595195] ata6: failed to resume link (SControl FFFFFFFF)
[   78.230476] ata6: SATA link down (SStatus FFFFFFFF SControl FFFFFFFF)
[   79.973077] ahci 0000:01:00.0: AHCI controller unavailable!
...

Running lspci is also somewhat slow and takes a couple of seconds to complete. lspci -vv:

0000:01:00.0 SATA controller: ASMedia Technology Inc. ASM1166 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
        Subsystem: ZyDAS Technology Corp. ASM1166 Serial ATA Controller
        !!! Unknown header type 7f
        Interrupt: pin ? routed to IRQ 43
        Region 0: Memory at 1b00080000 (32-bit, non-prefetchable) [size=8K]
        Region 5: Memory at 1b00082000 (32-bit, non-prefetchable) [size=8K]
        Expansion ROM at 1b00000000 [disabled] [size=512K]
        Kernel driver in use: ahci
        Kernel modules: ahci

Additional context

No response

[P33M]

You're running an Ubuntu kernel that doesn't correspond to any release we've made for Raspberry Pi OS. Assuming you have a pciex1-compat-pi5.dtbo in your boot filesystem's /overlays directory, then add these lines to config.txt:

dtparam=pcie-32bit-dma-pi5
dtoverlay=pciex1-compat-pi5,no-mip

(and reboot).

[albertz]

Thanks for the response.

pciex1-compat-pi5.dtbo in your boot filesystem's /overlays directory

I have the file /boot/firmware/overlays/pcie-32bit-dma-pi5.dtbo. I assume this is different file? I don't have anything like *pciex* in my /boot.

I tried with dtoverlay=pcie-32bit-dma-pi5 but that does not seem to work.

I'm not sure how easy it is to update my firmware (to get that overlay)? apt update && apt upgrade is up-to-date already.

[6by9]

No typo. https://github.com/raspberrypi/linux/blob/rpi-6.6.y/arch/arm/boot/dts/overlays/pciex1-compat-pi5-overlay.dts

It was only merged 3 weeks ago, so you can either use sudo rpi-update (normal warnings apply over potential regressions), or download just that one overlay with wget https://github.com/raspberrypi/rpi-firmware/raw/master/overlays/pciex1-compat-pi5.dtbo and then sudo cp pciex1-compat-pi5.dtbo /boot/firmware/overlays

[albertz]

or download just that one overlay with wget https://github.com/raspberrypi/rpi-firmware/raw/master/overlays/pciex1-compat-pi5.dtbo and then sudo cp pciex1-compat-pi5.dtbo /boot/firmware/overlays

Thanks! That's what I did now, and this seems to work!

[julienrobin28]

Hi, just as a feedback, also using a Serial ATA ASM1166 based PCIe card on which I had the same issue; Raspberry Pi 5, and P02 PCIe Slot Adapter Board from 52Pi, on latest and up to date Raspberry Pi OS.

Only added dtoverlay=pciex1-compat-pi5,no-mip to /boot/firmware/config.txt confirmed to work 👍

However, adding dtparam=pcie-32bit-dma-pi5 alone did nothing, and for now, it even works without it (previous option is enough). But is dtparam=pcie-32bit-dma-pi5 still desirable for any reason in this case?

I still have a question about this msi-controller change. By default, we can see into bcm2712.dtsi from rpi-kernel 6.6.31 that the normal configuration is:

• pcie0 (pcie@100000) is using pcie0 msi-controller
• pcie1 (pcie@110000) is using mip1 msi-controller (msi-controller@131000)

When using dtoverlay=pciex1-compat-pi5,no-mip

• pcie0 (pcie@100000) is using pcie0 msi-controller
• pcie1 (pcie@110000) is using pcie1 msi-controller

But is there any drawbacks / loss of possible features doing so? 🤔 Is the default configuration using mip1 msi-controller for pcie1 relevant for any features or resource optimization? My insufficient knowledge about this isn't allowing me to figure it out.

Thank you in advance for any explanations on this subject, hoping this may help people that would also be googling for information about this.

[pelwell]

There will be a theoretical performance penalty because there is an extra level of software decode required to get to the relevant interrupt handler, but it may not be measurable.

[owendswang]

No typo. https://github.com/raspberrypi/linux/blob/rpi-6.6.y/arch/arm/boot/dts/overlays/pciex1-compat-pi5-overlay.dts

It was only merged 3 weeks ago, so you can either use sudo rpi-update (normal warnings apply over potential regressions), or download just that one overlay with wget https://github.com/raspberrypi/rpi-firmware/raw/master/overlays/pciex1-compat-pi5.dtbo and then sudo cp pciex1-compat-pi5.dtbo /boot/firmware/overlays

It seems not working now. It couldn't boot up with Ubuntu Server 24.04 with latest updates. I downloaded 'pciex1-compat-pi5.dtbo' to '/boot/firmware/overlays/'. Here is my 'config.txt' ending lines:

[all]

# dtparam=pciex1
# dtparam=pciex1_gen=3

dtparam=pcie-32bit-dma-pi5
dtoverlay=pciex1-compat-pi5,no-mip