Pi4 crash when using 2nd USB3 device when booting from USB3 SSD

[poliboy]

I use the Pi4 booting with a USB3 connected SSD. This works nicely, but when I connect in addition an external USB3 HDD to the USB3 port, then the system crashes with "IO errors". Any commands like "dmesg" are not available any longer. Connection of the HDD to USB2 port works. Booting from SD card and connecting the USB3 SSD and the USB3 HDD at the same time works.

Actual behaviour The GUI disappears (only background is visible). Some "IO errors" are indicated.

System System Information

Raspberry Pi 4 Model B Rev 1.1 PRETTY_NAME="Raspbian GNU/Linux 10 (buster)" NAME="Raspbian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)"

Raspberry Pi reference 2019-09-26 Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 80d486687ea77d31fc 3fc13cf3a2f8b464e129be, stage5

Linux toni4 4.19.97-v7l+ #1294 SMP Thu Jan 30 13:21:14 GMT 2020 armv7l GNU/Linux Revision : c03111 Serial : 100000000824fe4c Model : Raspberry Pi 4 Model B Rev 1.1 Throttled flag : throttled=0x0 Camera : supported=0 detected=0

• Which model of Raspberry Pi? Pi4

• Which OS and version (cat /etc/rpi-issue)? Raspberry Pi reference 2019-09-26 Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 80d486687ea77d31fc3fc13cf3a2f8b464e129be, stage5

• Which firmware version (vcgencmd version)? Feb 12 2020 12:36:21 Copyright (c) 2012 Broadcom version c3c8dbdf147686fb0c3f32aece709d0653368810 (clean) (release) (start)

• Which kernel version (uname -a)? Linux toni4 4.19.97-v7l+ #1294 SMP Thu Jan 30 13:21:14 GMT 2020 armv7l GNU/Linux

Logs unfortunately demsg is not available any longer

Additional context Add any other relevant context for the problem.

[tiagoboldt]

Have you tried connecting the USB disks using a powered USB hub? The pi4 might now have enough juice to power 2 external disks. I've struggled with this in the past and solved it with a powered hub. Not sure how this relates to it working on the USB2 ports, not sure if the USB3 port is pulling more power.

[poliboy]

Good point. The SSD and its USB3 interface are bus powered and working standalone when booting from it. The additional USB3 HDD has a power supply attached. My expectation is, that there is not more current drawn from USB3 port when plugging in a powered device. I tried with a powered USB3 hub and both disks worked also in USB3 boot mode. I assume that the driver software used is a different one, because only one USB3 port from the Pi4 is used. Unfortunately I can't check with a non-powered USB3 hub. Therefore: I'm pretty sure that there is no issue with the power consumption, but full evidence is missing.

[popcornmix]

The additional USB3 HDD has a power supply attached. My expectation is, that there is not more current drawn from USB3 port when plugging in a powered device.

I don't think that is a valid expectation. Often the power supply to a drive just supplies, say 12V for the motor. There may be significant 5V still drawn from the Pi over USB.

[poliboy]

Thanks for your replies so far. In order to remove all doubts, I purchased a passive USB3 hub (therefore my answer took some time). The result: If I attach the passive hub to one USB3 port of the Pi4, and the SSD to the passive hub, then it works (as expected). If I then attach the HDD to the passive hub, it works as well! If I instead attach the HDD to the 2nd USB3 port of the Pi4, then the crash happens. This is reproducible also with another HDD (I used Seagate and WD). I consider this as evidence that the problem is not the too high current drawn from the Pi4. Some other experiment: I booted with SSD directly in USB3 port of Pi4, and after plugging in the passive USB hub, then the crash happened. Nothing was attached to the passive hub. Let me know if I could support the bug tracing by some SW analytics...

[tiagoboldt]

I don't think there's any bug here to be solved. It's an electronic limitation. When you draw more current then the pi can provide, it restarts. No way around that. Maybe this should be better documented -- this issue might do just that for those who have the same problem in the future.

I guess this can be closed, but I'll leave that feedback to @popcornmix who's the expert here.

[poliboy]

Maybe my formulation was too complicated: I don't draw more current than the Pi can provide... Otherwise I would see the same issue with the passive hub, which is not the case.

[P33M]

Plug an optical mouse into one of the spare USB ports. When the crash occurs, does the LED on the bottom of the mouse turn off?

[poliboy]

Thanks for the hint. I attached this: Bottom USB3: SSD Bottom USB2: optical mouse After booting, everything was fine. Then I plugged the SSD (actively supplied) into the Top USB3. The crash occurred, but the optical mouse still was working perfectly and the LED was still on.

[poliboy]

I did another experiment, maybe this refers better to your question: Starting with Bottom USB3: SSD I booted without mouse and keyboard. Then I plugged the optical mouse into the Top USB3. The crash happened, the mouse LED was on, but the mouse didn't work.

[P33M]

Do you have a powered USB3.0 hub, and if so, what is the behaviour with one or both SSDs plugged into the hub?

[poliboy]

I have a powered USB3 hub (combined with a HDD). If I plug this into the Pi4 instead of the passive hub, and attach the drives to the powered hub, then everything works fine.

I did some more investigations on the issue and found out, that a message appears at the boot screen, which was not visible in dmesg. It took several trials to take a photo at the right point in time. The message said: "Failed to start persistent change of wireless mouse interrupt to avoid conflict with USB3 SSD. See 'systemctl status mouse_irq.service' for details." The message didn't disappear when I used a wired mouse and keyboard.

The output of the mentioned "systemctl" command was:

● mouse_irq.service - persistent change of wireless mouse interrupt to avoid conflict with USB3 SSD
   Loaded: loaded (/etc/systemd/system/mouse_irq.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sat 2020-03-28 03:32:50 CET; 8h ago
  Process: 519 ExecStart=/bin/bash -c echo 2 | sudo tee /proc/irq/55/smp_affinity (code=exited, status=1/FAILURE)
 Main PID: 519 (code=exited, status=1/FAILURE)

Mär 28 03:32:49 toni4 systemd[1]: Starting persistent change of wireless mouse interrupt to avoid conflict with USB3 SSD...
Mär 28 03:32:50 toni4 sudo[523]:     root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/bin/tee /proc/irq/55/smp_affinity
Mär 28 03:32:50 toni4 sudo[523]: pam_unix(sudo:session): session opened for user root by (uid=0)
Mär 28 03:32:50 toni4 bash[519]: tee: /proc/irq/55/smp_affinity: Datei oder Verzeichnis nicht gefunden
Mär 28 03:32:50 toni4 bash[519]: 2
Mär 28 03:32:50 toni4 sudo[523]: pam_unix(sudo:session): session closed for user root
Mär 28 03:32:50 toni4 systemd[1]: mouse_irq.service: Main process exited, code=exited, status=1/FAILURE
Mär 28 03:32:50 toni4 systemd[1]: mouse_irq.service: Failed with result 'exit-code'.
Mär 28 03:32:50 toni4 systemd[1]: Failed to start persistent change of wireless mouse interrupt to avoid conflict with USB3 SSD.

"Datei oder Verzeichnis nicht gefunden" is translated: "File or directory not found".

Maybe this is related to the issue I described at the beginning and you know what's going on there...

[m-t-smith]

Hey Poliboy,

Did you ever determine the exact issue? I experienced a similar problem; I booted and ran an HP laptop from a USB3 attached external SSD Ubuntu 20.04 installation. This setup worked fine until I plugged a second SSD into the other USB3 port, at which point Ubuntu crashed and I got a black screen full of a rapidly repeating I/O error. If this is the same problem, I'm glad to hear that using a USB2 port for one of the drives or booting from an SD card works (though I'm unsure is this means you're booting the OS on your SSD with a bootloader on your SD card, or booting fully from the SD card and leaving the SSD-based OS to languish.).

I plan to use the SSD-based OS on a pi4, but have you tried recreating the issue on any other type of computer? I share your suspicion that something besides power is causing the problem, though that's probably usually a fair bet for a pi, and hence why everyone seems so hung up on it even after you ran both through a passive hub. My gut feeling is that the problem lies with the usb3 driver (probably UAS) integrated into the Linux kernel. USB3s cause interference; I'm guessing part of the solution to that was a bit of a hack job that involved blocking all/the other USB3 ports temporarily whenever something new was plugged into one. That would obviously be a problem for this kind of setup, because it cuts of the OS, so the interrupt handler is never reached and the request just keeps running because it's supposed to be non-blocking.

That's just my guess, would like to hear if you or anyone else figured it out.

[clssn]

Hi everyone, looks like I have a very similar issue, though I don't boot from the SSD (only have /var mounted from one of them, which makes a crash equally destructive). I initially thought it was power related but after some weeks of trying around similar things as @poliboy power doesn't appear to be the problem. In my setup the crash happens after a couple of hours of flawless operation which makes it harder to inspect, together with journal disappearing with /var. For now, I just wanted to confirm the problem. I'll check next, whether using only the bottom USB3 port works for me as well, and whether by plugging in arbitrary devices in the top port I can provoke the crash. @m-t-smith, according to what you tell, it's about the point in time of connection, while I don't connect anything new when the crash happens.

Good to know I'm not alone, though it's a bit quiet in here while I can't believe we're the only ones with that particular use-case and problem.

Edit: Looks like I finally found my problem. It seems to be well known (see for example #3070) that some JMicron USB3 chipsets cause problems like mine and can be circumvented using a "quirk". Searching for the term "usb uas JMicron" there's a lot more information about the incompatibility (both Linux and Windows seems to be affected). Shopping for USB3 adapters you should search for UASP compatible devices - or disable UASP with the quirks.

[sourabhjaiswal]

Hi,

I am also facing the same problem. My setup is :- Raspberry pi 4 4gb USB SSD: https://www.amazon.in/gp/product/B08N187N6N/ USB HDD: https://www.amazon.in/gp/product/B01N07NBLA

I am booting system with USB SSD and connecting HDD later on. but as soon as I connect the HDD it crashes. I tried with one Seagate HDD too but same result.

Any feedback on resolving it would be greatly appreciated.

[ghost]

Hi,

I am also facing the same problem. My setup is :- Raspberry pi 4 4gb USB SSD: https://www.amazon.in/gp/product/B08N187N6N/ USB HDD: https://www.amazon.in/gp/product/B01N07NBLA

I am booting system with USB SSD and connecting HDD later on. but as soon as I connect the HDD it crashes. I tried with one Seagate HDD too but same result.

Any feedback on resolving it would be greatly appreciated.

Use a decent quality powered hub. That applies to anyone with USB problems with drives attached, even just one drive.

An example of a decent quality powered USB 3.0 hub is the D-Link DUB-1340, which is what I use and it works great. See https://eu.dlink.com/uk/en/products/dub-1340-4-port-superspeed-usb-3-hub.

The rest of the problems with using USB drives with the Pi can be traced to poor quality drives and enclosures - usually it's the enclosure. I recommend startech.com enclosures. There are very few problems which can be attributed to the Pi itself.

[xMadNet]

Hi, Same issue here A Pi 4 8Gb, update, upgrade done, Rpi-update done Linux XPi4 5.10.33-v7l+ #1415 SMP Fri Apr 30 15:50:57 BST 2021 armv7l GNU/Linux I have a mSata linked with a x857 card. I Boot on this ssd Everything fine I bought an ORICO 9548RU3 HDD Enclosure (so externally powed) plug on the remaining USB3 It usually works.... But sometimes (once a day) I have those IO errors (ls, df ... returns errors) After a hard reboot all is fine If I plug in the USB2 it works fine (but half hard disk throughput :-( ) Any ideas ? BR

[xMadNet]

Hi, To solve my problem I've connected booting SSD tu UBS2 and HHD enclosure to USB3 (only this device on USB3) No issues with this configuration ! So I don't understantd the link made with Poor Quality HDD Enclosure. BR

[o3o3o]

Hi, thanks all for sharing the issue here.

Same issue for pi4b 8G. I used two old HDDs with USB3. There is an HDD connected with a USB3.0 Y cable to supply the basic voltage.

The HDDs crash sometimes when there are large data writes into one of the HDD, because I am running a Bitcoin node to sync IBD with PI 4B. Here is a piece of logs when the disk started to crash.

May 12 19:59:44 pi4b kernel: [17074.552508] sd 0:0:0:0: [sda] tag#0 data cmplt err -75 uas-tag 1 inflight: CMD
May 12 19:59:44 pi4b kernel: [17074.552520] sd 0:0:0:0: [sda] tag#0 CDB: Read(10) 28 00 08 28 03 00 00 01 00 00

May 12 20:00:16 pi4b kernel: [17106.390954] scsi host0: uas_eh_device_reset_handler start
May 12 20:00:16 pi4b kernel: [17106.519752] usb 2-1: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
May 12 20:00:16 pi4b kernel: [17106.542976] scsi host0: uas_eh_device_reset_handler success
May 12 20:00:46 pi4b kernel: [17136.787308] scsi host0: uas_eh_device_reset_handler start
May 12 20:00:46 pi4b kernel: [17136.789035] xhci_hcd 0000:01:00.0: WARNING: Host System Error
May 12 20:00:51 pi4b kernel: [17141.758517] sd 1:0:0:0: [sdb] tag#11 sense submit err -22 uas-tag 1 inflight: s-st a-out s-out a-cmd s-cmd
May 12 20:00:51 pi4b kernel: [17141.758531] sd 1:0:0:0: [sdb] tag#11 CDB: Write(10) 2a 00 1d 06 ed 68 00 00 50 00

May 12 20:00:51 pi4b kernel: [17141.899323] sd 1:0:0:0: [sdb] tag#9 CDB: Write(10) 2a 00 1d 06 ed 68 00 00 50 00
May 12 20:00:51 pi4b kernel: [17141.900460] sd 0:0:0:0: [sda] tag#0 CDB: Read(10) 28 00 01 d9 30 d8 00 00 18 00
May 12 20:00:51 pi4b kernel: [17141.900467] sd 0:0:0:0: [sda] tag#1 uas_zap_pending 0 uas-tag 3 inflight:
May 12 20:00:51 pi4b kernel: [17141.900473] sd 0:0:0:0: [sda] tag#1 CDB: Read(10) 28 00 08 28 04 00 00 01 00 00
 light:
May 12 20:00:51 pi4b kernel: [17141.900513] sd 0:0:0:0: [sda] tag#4 CDB: Write(10) 2a 00 0d 66 66 c0 00 00 08 00
May 12 20:00:51 pi4b kernel: [17141.900535] usb 1-1: USB disconnect, device number 2
May 12 20:00:51 pi4b kernel: [17141.900542] scsi host0: uas_eh_device_reset_handler FAILED err -22
May 12 20:00:51 pi4b kernel: [17141.900555] sd 0:0:0:0: Device offlined - not ready after error recovery
 

May 12 20:00:51 pi4b kernel: [17141.900542] scsi host0: uas_eh_device_reset_handler FAILED err -22
May 12 20:00:51 pi4b kernel: [17141.900555] sd 0:0:0:0: Device offlined - not ready after error recovery
 
May 12 20:00:51 pi4b kernel: [17141.901142] usb 2-1: USB disconnect, device number 2
May 12 20:00:51 pi4b kernel: [17141.907316] blk_update_request: I/O error, dev sda, sector 31010944 op 0x0:(READ) flags 0x80700 phys_seg 4 prio class 0
 
May 12 20:00:51 pi4b kernel: [17141.929253] EXT4-fs warning (device sda): ext4_end_bio:309: I/O error 10 writing to inode 20323581 (offset 0 size 0 starting block 28101849)
May 12 20:00:51 pi4b kernel: [17141.929269] Buffer I/O error on device sda, logical block 28101848
May 12 20:00:51 pi4b kernel: [17141.935616] blk_update_request: I/O error, dev sda, sector 136839936 op 0x0:(READ) flags 0x80700 phys_seg 32 prio class 0
 
May 12 20:00:51 pi4b kernel: [17141.991456] blk_update_request: I/O error, dev sda, sector 224815816 op 0x1:(WRITE) flags 0x4000 phys_seg 128 prio class 0

May 12 20:00:51 pi4b kernel: [17142.024124] Buffer I/O error on device sda, logical block 28101849
 
May 12 20:00:51 pi4b kernel: [17142.074519] Buffer I/O error on device sda, logical block 28101857
May 12 20:00:51 pi4b kernel: [17142.082431] EXT4-fs warning (device sda): ext4_end_bio:309: I/O error 10 writing to inode 20323581 (offset 21860352 size 3305472 starting block 28102391)

  
May 12 20:01:02 pi4b dockerd[1834]: time="2021-05-12T20:01:02.327205402+08:00" level=error msg="3702dc017908460f14cae978219e317743def1c33457c21eda9153d785c4543d: failed saving state on start failure: lstat /mnt/home/docker_data/docker/containers/3702dc017908460f14cae978219e317743def1c33457c21eda9153d785c4543d/config.v2.json: input/output error"
May 12 20:01:02 pi4b systemd-udevd[21581]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
May 12 20:01:02 pi4b dockerd[1834]: time="2021-05-12T20:01:02.327501492+08:00" level=warning msg="failed to get endpoint_count map for scope local: open /mnt/home/docker_data/docker/network/files/local-kv.db: input/output error"
...

lsusb -t

/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=uas, 5000M
    |__ Port 2: Dev 4, If 0, Class=Mass Storage, Driver=uas, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M

Updated: May, 15 The reason seems that I need to get enough voltage and current to power two HDDs. I think it is the same as 2nd SSD. The max export current is 1.2A for pi4. It didn't work well although I had powered only one HDD with an extra power adapter. So I bought a powered USB cable and going to have a try.

[daedalia]

I've had some similar issues. This article provides some insight and good suggestions. https://www.pragmaticlinux.com/2021/03/fix-for-getting-your-ssd-working-via-usb-3-on-your-raspberry-pi/

[schmidchr]

I've had some similar issues. This article provides some insight and good suggestions. https://www.pragmaticlinux.com/2021/03/fix-for-getting-your-ssd-working-via-usb-3-on-your-raspberry-pi/

thanks to @daedalia , disabling UASP ('Fix 2' in above article) enabled me to have two 'active devices' (SSD) in raid1. Before the second SSD was reported 'removed' by mdam -D /dev/md1
As indicated in the article, hopefully a software or firmware fix becomes available in the not too distant future, to get that extra 10-30% that the UASP protocol could squeeze out.

[KeenMaron]

Hi! I have the same problem, as soon as I connect a second drive (with external power) the GUI of raspberian goes black and nothing is clickable anymore on the desktop. I will try it with an active hub, but it's weird, that the external power of the second hard drive is not enough??

My setup (not working with both drives attached; works only with SSD only): raspberrypi 8GB + official power supply USB 3.0 SSD with external power <- systeminstallation USB 3.0 HDD with external power <- data

[JohannesArtmeier]

I have some similar issues with Ubuntu Server 22.04, but its happing also on USB 2.0 ports. Here you can find my StackOverflow post.

[peterstamps]

A lot of issues with powered USB 3.0 hubs, Raspberry Pi 4 and a HDD/SSD with a SATA-USB 3.0 cable or in a Case may be solved by adding a command into your /boot/cmdline.txt file of your Raspberry Pi 4.

This solution solved everything for me!

Read here how you can fix that: https://www.pragmaticlinux.com/2021/03/fix-for-getting-your-ssd-working-via-usb-3-on-your-raspberry-pi/#:~:text=For%20some%20reason%20the%20Raspberry%20PI%204%20does,connecting%20your%20SSD%20to%20your%20Raspberry%20PI%204.