tinyusb icon indicating copy to clipboard operation
tinyusb copied to clipboard

Enhance HITL (hardware in the loop) test

Open HiFiPhile opened this issue 2 years ago • 16 comments

Related area

HITL test

Hardware specification

DCD only for the moment

Is your feature request related to a problem?

I'm going to add more boards into the loop, currently planned:

  • LPCXpresso LPC54608 OM13092 (HS port)
  • STM32F746DISCO (both HS & FS port)
  • NUCLEO-L412KB (like now)

Meanwhile maybe a good idea to wait #1970 to avoid mixing boards.

I'm also open to sponsorship if anyone wants other boards to be added.

PS: Made a tiny hub as I can't find one :) IMG_20230731_194054

Describe the solution you'd like

Add more boards into HITL setup.

I have checked existing issues, dicussion and documentation

  • [X] I confirm I have checked existing issues, dicussion and documentation.

HiFiPhile avatar Jul 31 '23 17:07 HiFiPhile

thank you, that is great to hear, I can implement #1970 soon enough, your current list of board is great, it already cover 3 different usb ips. I have an Pi4 with pico rp2040 running tests and plan to add more boards (since I have several devkit laying around), since these aren't computing heavy as compiling IAR :).

We will probably need to spend more time to write more actual hardware tesing script :) . Only tested cdc, msc, dfu so far. Haven't tried testing with audio.

PS: what's up with the hub breakout, wouldn't it look nicer with consumer hub :)

hathach avatar Aug 01 '23 09:08 hathach

We will probably need to spend more time to write more actual hardware testing script

Yes I'm also thinking about it.

what's up with the hub breakout, wouldn't it look nicer with consumer hub :)

For space saving, with male plug directly soldered on the hub I don't need to deal with cable mess ;)

HiFiPhile avatar Aug 01 '23 09:08 HiFiPhile

Here are a couple hardware project for HITL. Thanks @esden for the pointers! https://github.com/probe-rs/hive https://github.com/esden/redcuttle

tannewt avatar Aug 01 '23 16:08 tannewt

Here are a couple hardware project for HITL. Thanks @esden for the pointers! https://github.com/probe-rs/hive https://github.com/esden/redcuttle

thanks Scott, it looks very neat, but probably overkilled I guess. I would prefer to just stick the dev board in, most of them has on-board debugger anyway for flashing. It is of course just me :D

hathach avatar Aug 03 '23 08:08 hathach

#2204 implement unique ID for several families including: rp2040 and various stm32 (including l4 and f7)

hathach avatar Aug 04 '23 03:08 hathach

I've connected the boards.

3 debuggers are flashed to Jlink:

SN Board Device
770935966 STM32F746DISCO STM32F746NG
774470029 NUCLEO-L412KB STM32L412KB
727600775 OM13092 LPC54608J512

Both STM32L4 & STM32F7 works, while only HS port works on OM13092. (added missing -DBOARD_TUD_RHPORT=$(PORT))

Tried changing 54628 to 54608 in board.mk and I don't spot any USB related difference...

HiFiPhile avatar Aug 04 '23 09:08 HiFiPhile

thank you :+1: , just saw them show up in the lsusb (ssh shell). I will try to update ci test script later on. Since you are often working with audio, maybe we should spend more time to write some script to capture the mic input and compare with knowngood etc... but that can always wait :)

hathach avatar Aug 04 '23 11:08 hathach

Since you are often working with audio, maybe we should spend more time to write some script to capture the mic input and compare with knowngood etc... but that can always wait :)

It would be nice but compare audio is a little tricky due to PC resampling etc., I'll try to figure something out.

HiFiPhile avatar Aug 04 '23 12:08 HiFiPhile

I've added a LPCXpresso4367 to replace 54608, it passes all tests with EA4357 BSP. I think it's good to have chipidea_hs tested.

PS1: Can we modify Make to build both FS & HS option for stm32f746 ? Since HS use external phy I think it worth to test both.

PS2: Potential null pointer dereference building host example for ea4357.

/home/tusb/tinyusb/src/portable/ehci/ehci.c: In function 'hcd_edpt_xfer.constprop':
/home/tusb/tinyusb/src/portable/ehci/ehci.c:459:5: error: potential null pointer dereference [-Werror=null-dereference]
  459 |     TU_VERIFY(!qhd->qtd_overlay.halted);
      |     ^
compilation terminated due to -Wfatal-errors.

qhd_get_from_addr() could return NULL.

HiFiPhile avatar Aug 11 '23 10:08 HiFiPhile

ah thanks, I also have LPCXpresso43s67, I can help adding bsp and testing this out.

Can we modify Make to build both FS & HS option for stm32f746 ? Since HS use external phy I think it worth to test both.

Yeah, sure, I would love to, we will need

  • add addtional json field for compile argument e.g PORT=0 (may need to duplicate f476 in json)
  • update test scrip to build to build with extra compile argument and run test on f746

You don't have to do this, give me a bit of time, I will try to see what would be the best way to implement this.

Potential null pointer dereference building host example for ea4357.

weid, I don't seems to see that warnings, I will check it out later.

hathach avatar Aug 14 '23 06:08 hathach

#2217 add lpc43s67 since it is what I got. I guess it is probably run fine with your non-s board. If it is not the case we can change the jlink device to non-s verion. Also add uuid support lpc43. Let me know if those are now working with your hil setup there :)

hathach avatar Aug 14 '23 10:08 hathach

A type-C switcher allows to programmatically connect two devices pin-by-pin, rather than using a HUB). Some USBC0 to USBC1, or some USBC0 to USBC2, either at a time. https://www.digikey.com/en/blog/introducing-the-ultimate-usb-type-c-switch https://www.codethink.co.uk/articles/2019/codethink-creates-custom-usb-3-switch-to-support-customers-cicd-pipeline-requirements/

This can help to automate host-mode operation:

  1. Workstation <-> DUT: Load a firmware to the DUT (in device mode)
  2. Keyboard <-> DUT: Test the DUT keyboard support (in host mode) and write the test results to flash
  3. Workstation <-> DUT: Get the results from the flash over USB (in device mode)

josuah avatar Jan 14 '24 20:01 josuah

@josuah

Thanks for your interest in HIL test.

A port switcher is not the priority, as currently all boards has a debugger attached who can read test results.

Currently all boards in the loop are bought by me or hathach, without a sponsorship we can't test all kinds of configurations.

HiFiPhile avatar Jan 14 '24 20:01 HiFiPhile

Thinking again, if wanting to leave all boards connected at all time for an automated test script, a switcher of only the D+/D- and +5V lines would be a good compromise.

A port switcher is not the priority, as currently all boards has a debugger attached who can read test results.

Noted! This sounds like a better solution indeed.

For practical setup, regardless of the solution used, here is something that scales well and is cheap and quick to do:

Testing_USB_devices_practical_compact_setup

A bit like a thick wallet of PCBs. Or an office binder made for PCBs. Of course, everyone will have their own method and preference according to their material at hand and preferences!

Without a sponsorship we can't test all kinds of configurations.

Several projects might want a HITL test rig. As discussed on 1BitSquared Discord, there are several test project slowly getting built.

  • https://github.com/esden/redcuttle
  • https://github.com/probe-rs/hive
  • https://renode.readthedocs.io/en/latest/introduction/supported-boards.html
  • that project without a name discussed on the chat 1, 2
  • [...]

It looks like a solution is already engineered-out at TinyUSB.

josuah avatar Jan 14 '24 21:01 josuah

I think I have never showed how it's done:)

One 1 to 7 hub PCB and 3 boards separated by paper inside a cardboard box :

IMG_20240114_221849917

HiFiPhile avatar Jan 14 '24 21:01 HiFiPhile

@josuah thank you for your suggestion, I only have plan to put each dev board to a plastic bag and try to push it under my desk as many as I could.

@HiFiPhile your setup is still much better than mine, just an RPI4 + bunches of dev board in plastic bags packed under my PC desk.

IMG20240115234240

PS: IAR build failed due to expired license, I have contact them to renew our opensource license.

hathach avatar Jan 15 '24 16:01 hathach