dasharo-issues icon indicating copy to clipboard operation
dasharo-issues copied to clipboard

Spurious USB 3 disconnects with Sonnet Echo 11 Thunderbolt 4 dock

Open SebastianCzapla opened this issue 1 year ago • 13 comments

Component

Dasharo firmware

Device

NovaCustom V54 14th Gen

Dasharo version

v0.9.1-rc6

Dasharo Tools Suite version

No response

Test case ID

No response

Brief summary

When connected to backports of docking station, USB drives randomly disconnect

How reproducible

75% windows, disconnects occur every 10-15 seconds.

How to reproduce

  • Boot to Windows 11
  • Disconnect monitors from dock, if present
  • Place a USB drive into backport of Echo11
  • Attempt an operation on that drive format / file write.

Expected behavior

USB drive stays mounted

Actual behavior

On windows, USB drive disconnects and reconnects, error message shows that a target device of file operation does not exists.

Screenshots

No response

Additional context

Only happens on Win11 when no monitors are connected to the dock. Only happens with this dock model, does not happen with USB Type-C DP Alt mode docks like Wavlink UMD05 Pro Does not happen on OEM Insyde firmware on the same laptops.

Solutions you've tried

Workaround is to attach USB-C DisplayPort cable to Echo11 dock, then the issue goes away, with 100% success.

Connecting dock to another PC and then reconnecting to DUT seems to "revive" USB storage capability of dock, but not always.

SebastianCzapla avatar Oct 08 '24 10:10 SebastianCzapla

Issue still present in v0.9.1-rc7, affects Adata/Sandisk/Kingston drives

SebastianCzapla avatar Oct 17 '24 09:10 SebastianCzapla

Within windows, issue has escalated a bit: whole dock looses link with DUT after some time (hard to tell, but seems like 5 minutes or more)

On rc6, it affected only pendrives, USB keyboard connected to dock was working at all times. Now with rc7, whole dock looses connection, and attached devices stop working. This happens (sometimes) when rebooting and going into UEFI settings. So far, I haven't noticed the same behavior on Linux but it might change.

Edit: On Windows, this issue also affects Ethernet connection, SD Card reader, USB drives attached to the dock. Keyboard and external headset seems to work most of the time, even after other devices are dropped due to disconnect.

SebastianCzapla avatar Oct 17 '24 13:10 SebastianCzapla

Another observation on rc7 on Windows: Issue reproduction rate falls almost to 0 when USB-C Display Port is attached to the docking station. USB Drives do not disconnect, Ethernet does not drop. Disconnecting USB-C DP almost immediately drops ethernet connection.

SebastianCzapla avatar Oct 21 '24 13:10 SebastianCzapla

@SebastianCzapla rc8 has improvemtns for this, but please make sure you're doing tests with the 180W charger connected to the system, that's important for stability

mkopec avatar Oct 28 '24 11:10 mkopec

On rc8 (ec and FW), with fresh install of Windows11 the issue of random USB drive disconnects is still present.

Laptop powered via 180W adapter, docking station has only a usb pendrive.

Every few seconds, pendrives disconnect and any file operation is broken. Same happens for Ethernet port of the docking station.

Once again, issue goes away, when a USB-C DisplayPort cable is attached. When the screen connected via USB-C-DP goes to sleep, issue comes back.

SebastianCzapla avatar Oct 29 '24 14:10 SebastianCzapla

Happens on V560TNE on Dasharo but not on Insyde. Looks like a Dasharo bug.

mkopec avatar Oct 30 '24 12:10 mkopec

Seems to also depend on model of the USB stick being tested

  • Fangxiang stick does not disconnect (maybe because it's usb2 only)
  • Goodram stick drops out

also not all devices in the dock drop out. Ethernet (USB3) drops out, but USB Audio (USB2) does not.

So it looks like only USB3 is failing intermittently

mkopec avatar Oct 31 '24 17:10 mkopec

Issue still occurs on the https://github.com/Dasharo/coreboot/actions/runs/13854001267 version of Firmware. USB and Ethernet connection drops as before.

SebastianCzapla avatar Mar 14 '25 11:03 SebastianCzapla

Tested with INSYDE FIRMWARE on V540TU & V560TNE;

Windows 11: no reproduction Ubuntu 24: no reproduction

matmacieje avatar Apr 07 '25 10:04 matmacieje

Tested od Dasharo v1.0.0-rc1

Windows 11: issue exists, reproduction rate strongly depends on ethernet connection via Echo 11 port, formatted filesystem size (at least several gigabytes), and avoiding Quick Format option. Ubuntu 24: no reproduction.

matmacieje avatar Apr 07 '25 16:04 matmacieje

Could not reproduce anymore on https://github.com/Dasharo/ec/pull/71

mkopec avatar Apr 08 '25 14:04 mkopec

Issue does not exist anymore on Dasharo rc 2, tested on:

  • V540TU Windows 11
  • V540TU Ubuntu 24
  • V560TNE Ubuntu 24

Issue exists on Dasharo rc2, tested on:

  • V560TNE Windows 11
Image

matmacieje avatar Apr 09 '25 14:04 matmacieje

still occurs on V560TU when no monitor is connected to the dock

mkopec avatar Apr 09 '25 17:04 mkopec

Things I did attempting to fix the problems of the dock resetting under Windows:

  1. Gathered traces from the USB drivers using these methods:

https://learn.microsoft.com/en-us/windows-hardware/drivers/usbcon/how-to-capture-a-usb-event-trace https://learn.microsoft.com/en-us/windows-hardware/drivers/usbcon/how-to-install-netmon-and-the-netmon-usb-parser https://learn.microsoft.com/en-us/windows-hardware/drivers/usbcon/how-to-examining-a-trace-file-by-using-netmon

From these logs we could filter out events that happened on the Thunderbolt USB controller. There are some errors happening on the ThunderBolt USB HUB which lead to the port to go into an inactive state. Per the USB 3.x specification (https://usb.org/sites/default/files/usb_32_202206_0.zip) a port my end up in such state for one of couple reasons:

10.3.1.4
DSPORT.ERROR
A port shall transition to this state only when a n Enhanced SuperSpeed device is connected
and a serious error condition occurred while attempting to operate the link.
A port transitions to this state in any of the following situations:
•From the DSPORT.Enabled state if the link enters recovery and times out without
recovering.
•From the DSPORT.Enabled state if U1 or U2 exit fails.
•From the DSPORT.Loopback state if the port is the loopback master and the LFPS
handshake in Loopback.Exit fails.
•From DSPORT.Enabled if Port Configuration fails as described in Section 8.4.6.
•From the DSPORT.Training state if the port’s link times out from any Polling substate
and cPollingTimeout is 2. See 7.5.4.2 for details of cPollingTimeout.
In this state, the port’s link shall be in the eSS.Inactive state.

But still it didn't give us much clues what may be wrong.

  1. One clue from above was the U1 or U2 exit fails . We noticed that Intel’s reference code disable transitions to U1 and U2 via ACPI for Type-C ports. We did the same, but to no avail.
  2. Looped through multiple Intel documents how to properly set up the Type-C ports and Thunderbolt on the firmware side. Checked each requirement twice and found some misconfigurations on our side: https://github.com/Dasharo/coreboot/pull/662 But still it made no effect.
  3. Tried to update Thunderbolt retimer firmware, but the Thunderbolt port started to behave worse when I did it.
  4. Tried Insyde firmware but with our ME image to exclude a potential mismatch of host Thunderbolt blos with retimer firmware. But Insyde firmware was still unaffected, so the Thunderbolt blobs are not at fault here.
  5. Tried Dasharo with Insyde EC firmware to exclude a possibility of missing code on EC side. However, even with Insyde firmware the issue was present. But it doesn;t mean Insyde BIOS does not talk to EC about the Type-C connection...

I am out of ideas what else I may try to tackle this issue. The dock is reset and re-enumerated every 25s (regularly with a stopwatch in the hand), which may indicate some timeout. Not yet sure what may be causing this.

Summing it up, it is very likely that we miss something on the coreboot side. I have pretty much excluded other culprits here. It may be the beloved ACPI (I have already added pretty much a lot of ACPI code which seemed relevant, but still no effect)…

miczyg1 avatar May 16 '25 12:05 miczyg1

@miczyg1 I think Thunderbolt technology will evolve, where we could find professional hardware related forum for Thunderbolt controllers? It looks like Thunderbolt not going anywhere, so maybe joining https://www.thunderbolttechnology.net/developer-application would be something we should consider?

pietrushnic avatar May 19 '25 07:05 pietrushnic

Probably related: https://github.com/Dasharo/dasharo-issues/issues/1100

filipleple avatar Jul 08 '25 14:07 filipleple