EspTinyUSB
EspTinyUSB copied to clipboard
chegewara
USB MIDI Host [feature request]
What would need to be done to add MIDI USB Host support ...I have MIDI kit, time, patience, reasonable programming skillz, and very little knowledge about low-level USB.
There is few steps that has to be done before start:
- learn and understand MIDI descriptor(s); it is first step because its how midi device can be detected
- learn about endpoints in midi devices; i dont know how is it solved in midi, if each jack is a separate endpoint, or there is only 2 endpoints and jacks are distinguished in data
- by design, each host device should extend this class: https://github.com/chegewara/EspTinyUSB/blob/master/src/host/common/usb_device.cpp
I am not saying that my design is best or even optimized, but you can study existing examples.
MIDI is essentially a USBv1 (not v2) Audio Control Interface. Standard cables (random cheap example) provide simply USB <--> TWO 5-pin DIN sockets ...One is UART Tx [MIDI Out] ...the other is UART Rx [MIDI In]
FYI. Both UARTS are 31250Hz [1KHz/32] 8N1 @ 5v0, so they can be driven by pretty much any UART (maybe with level shifters) ...but more and more (consumer grade) kit is moving to USB.
The USB spec also defines "Xfr-In" and "Xfr-Out" Transfer Endpoints, but I have yet to find them anywhere in the real-world - so I've really not paid them any attention.
The serial adaptor chip presents MIDI In and MIDI Out as TWO (unidirectional) Bulk Endpoints ...and the packet size is FOUR bytes ...The Linux kernel has a hard-coded list of PID/VID's which it uses to patch the packet size to 4 (see line #1317) because many MIDI chips lie ...(and therefore don't work in Windows!)
Basically I have written [but obviously not debugged] everything** except the USB Transfer Mechanism:
- Act as virtual keyboard for a slave synth
- Act as synth for a slave keyboard
- Route MIDI Traffic
- Play MIDI files (MID, RIFF, format-0, and format-1 (so far))
My code (bar bugs) happily builds and/or decodes the 4-byte USB packets ...And I think I've got a pretty good API - but, obviously, it hasn't been abused yet, so there is undoubtedly room for improvement.
The problem I have is literally pushing and pulling the packets down the USB Bulk Endpoints.
I originally planned to use an RP2040, which implied the use of HaThach's TinyUSB project. But Ha advises me the RP2040 only has ONE Bulk Endpoint ...he has no support for the ESP32-S2 ...and no room in his workload to address this :-/
If your "design" can be made to work with MIDI, it will (IMHO) be "the best"! ...My only other option right now is a Teensy v3.6 (or possibly v4.1) ...They have a working MIDI Stack, but they are ~10x the price - so I will happily take "inelegant" working code over 'a dead project' :-)
Here is a dump I created (using Ha's code) of the Descriptors ...You can see the Bulk Endpoints being specified in Descriptors #11..14 ...I'd be interested to hear your thoughts :-)
Cheers, BC
vvv=================================================================vvv
~~~[01]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0009]--Rx::Descriptor
< 0000: 09 04 00 00 00 01 01 00 00
? [04] - TUSB_DESC_INTERFACE
? bLength = 9
? bDescriptorType = 0x04 (INTERFACE)
? bInterfaceNumber = 0
? bAlternateSetting = 0
? bNumEndpoints = 0
? bInterfaceClass = 0x01 (AUDIO)
? bInterfaceSubClass = 0x01 (AUDIO CONTROL)
? bInterfaceProtocol = 0
? iInterface = 0 ""
~~~[02]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0009]--Rx::Descriptor
< 0000: 09 24 01 00 01 09 00 01 01
? [24] - TUSB_DESC_CS_INTERFACE : AUDIO/AUDIO CONTROL
? bLength = 9
? bDescriptorType = 0x24
? bDescriptorSubType = 0x01 (AUDIO_DESC_CS_AC_INTERFACE)
? bcdADC = 0100 (bcd)
? wTotalLength = 9
? bInCollection = 1
? baInterfaceNr = {1}
~~~[03]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0009]--Rx::Descriptor
< 0000: 09 04 01 00 02 01 03 00 00
? [04] - TUSB_DESC_INTERFACE
? bLength = 9
? bDescriptorType = 0x04 (INTERFACE)
? bInterfaceNumber = 1
? bAlternateSetting = 0
? bNumEndpoints = 2
? bInterfaceClass = 0x01 (AUDIO)
? bInterfaceSubClass = 0x03 (MIDISTREAMING)
? bInterfaceProtocol = 0
? iInterface = 0 ""
~~~[04]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0007]--Rx::Descriptor
< 0000: 07 24 01 00 01 51 00
? [24] - TUSB_DESC_CS_INTERFACE : AUDIO/MIDISTREAMING
? bLength = 7
? bDescriptorType = 0x24
? bDescriptorSubType = 0x01 (MIDI_HEADER)
? bcdMSC = 0100 (bcd)
? wTotalLength = 81
~~~[05]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0006]--Rx::Descriptor
< 0000: 06 24 02 01 02 00
? [24] - TUSB_DESC_CS_INTERFACE : AUDIO/MIDISTREAMING
? bLength = 6
? bDescriptorType = 0x24
? bDescriptorSubType = 0x02 (MIDI_IN_JACK)
? bJackType = 0x01 (EMBEDDED)
? bJackID = 2
? iJack = 0 ""
~~~[06]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0006]--Rx::Descriptor
< 0000: 06 24 02 01 03 00
? [24] - TUSB_DESC_CS_INTERFACE : AUDIO/MIDISTREAMING
? bLength = 6
? bDescriptorType = 0x24
? bDescriptorSubType = 0x02 (MIDI_IN_JACK)
? bJackType = 0x01 (EMBEDDED)
? bJackID = 3
? iJack = 0 ""
~~~[07]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0006]--Rx::Descriptor
< 0000: 06 24 02 02 06 00
? [24] - TUSB_DESC_CS_INTERFACE : AUDIO/MIDISTREAMING
? bLength = 6
? bDescriptorType = 0x24
? bDescriptorSubType = 0x02 (MIDI_IN_JACK)
? bJackType = 0x02 (EXTERNAL)
? bJackID = 6
? iJack = 0 ""
~~~[08]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0009]--Rx::Descriptor
< 0000: 09 24 03 01 07 01 06 01 00
? [24] - TUSB_DESC_CS_INTERFACE : AUDIO/MIDISTREAMING
? bLength = 9
? bDescriptorType = 0x24
? bDescriptorSubType = 0x03 (MIDI_OUT_JACK)
? bJackType = 0x01 (EMBEDDED)
? bJackID = 7
? bNrInputPins = 1
? baSourceID = {6}
? baSourcePin = {1}
? iJack = 0 ""
~~~[09]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0009]--Rx::Descriptor
< 0000: 09 24 03 02 04 01 02 01 00
? [24] - TUSB_DESC_CS_INTERFACE : AUDIO/MIDISTREAMING
? bLength = 9
? bDescriptorType = 0x24
? bDescriptorSubType = 0x03 (MIDI_OUT_JACK)
? bJackType = 0x02 (EXTERNAL)
? bJackID = 4
? bNrInputPins = 1
? baSourceID = {2}
? baSourcePin = {1}
? iJack = 0 ""
~~~[10]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0009]--Rx::Descriptor
< 0000: 09 24 03 02 05 01 03 01 00
? [24] - TUSB_DESC_CS_INTERFACE : AUDIO/MIDISTREAMING
? bLength = 9
? bDescriptorType = 0x24
? bDescriptorSubType = 0x03 (MIDI_OUT_JACK)
? bJackType = 0x02 (EXTERNAL)
? bJackID = 5
? bNrInputPins = 1
? baSourceID = {3}
? baSourcePin = {1}
? iJack = 0 ""
~~~[11]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0009]--Rx::Descriptor
< 0000: 09 05 02 02 20 00 00 00 00
? [05] - TUSB_DESC_ENDPOINT
? bLength = 9
? bDescriptorType = 5
? bEndpointAddress = 0x02
? Direction +--[7 ] = 0 (out)
? Reserved +--[6:4] = 0
? Endpoint number `--[3:0] = 2
? bmAttributes = 0x02
? RESERVED +--[7:6] = 0
? usage type +--[5:4] = 0
? sync type +--[3:2] = 0
? xfer type `--[1:0] = 2
? wMaxPacketSize = 0x0020
? RESERVED +--[15:13] = 0
? hs_period_mult +--[12:11] = 0
? size `--[ 0:10] = 32
? bInterval = 0
? bRefresh = 0 (unused)
? bSynchAddress = 0 (unused)
~~~[12]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0006]--Rx::Descriptor
< 0000: 06 25 01 02 02 03
? [25] - TUSB_DESC_CS_ENDPOINT : AUDIO/MIDISTREAMING
? bLength = 6
? bDescriptorType = 0x25
? bDescriptorSubtype = 0x01 (MIDISTREAMING 'BULK' ENDPOINT)
? bNrEmbMIDIJack = 2
? baAssocJackID = {2,3}
~~~[13]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0009]--Rx::Descriptor
< 0000: 09 05 82 02 20 00 00 00 00
? [05] - TUSB_DESC_ENDPOINT
? bLength = 9
? bDescriptorType = 5
? bEndpointAddress = 0x82
? Direction +--[7 ] = 1 (in)
? Reserved +--[6:4] = 0
? Endpoint number `--[3:0] = 2
? bmAttributes = 0x02
? RESERVED +--[7:6] = 0
? usage type +--[5:4] = 0
? sync type +--[3:2] = 0
? xfer type `--[1:0] = 2
? wMaxPacketSize = 0x0020
? RESERVED +--[15:13] = 0
? hs_period_mult +--[12:11] = 0
? size `--[ 0:10] = 32
? bInterval = 0
? bRefresh = 0 (unused)
? bSynchAddress = 0 (unused)
~~~[14]~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
< [0005]--Rx::Descriptor
< 0000: 05 25 01 01 07
? [25] - TUSB_DESC_CS_ENDPOINT : AUDIO/MIDISTREAMING
? bLength = 5
? bDescriptorType = 0x25
? bDescriptorSubtype = 0x01 (MIDISTREAMING 'BULK' ENDPOINT)
? bNrEmbMIDIJack = 1
? baAssocJackID = {7}
^^^=====================================================================^^^
@csBlueChip
If you want to jumpstart the USB host engine to send/receive USB MIDI, you can find an example at https://github.com/touchgadget/esp32-usb-host-demos. The demo gets to the point where it displays 4 byte MIDI records on serial monitor from a USB MIDI keyboard. The demo ignores most of the audio descriptors.
Also demos for USB HID keyboard and USB printer exist. All are Arduino programs.
@touchgadget @csBlueChip
Did you manage to make the USB host to receive MIDI? Im planning to use a esp32-s3 with an additional usb breakout to make an external controller wireless via Bluetooth. Let me know your progress. Thanks
Sorry, i am not working on MIDI. Simply, because i dont know the protocol and i dont have any device which i could try to reverse engineer or to use as example.
Sorry, i am not working on MIDI. Simply, because i dont know the protocol and i dont have any device which i could try to reverse engineer or to use as example.
Thanks for your answer and for your work. I will try to understand the protocol with the available documentation on the usb forum, but i think is not for a beginner like me 😅
Sending and receiving data itself is not that hard, but if i want to make driver it would have to parse descriptors to handle In/Out jacks. I have to admit i have sometimes small problems to understand documents and i need hardware to work with.
Sending and receiving data itself is not that hard, but if i want to make driver it would have to parse descriptors to handle In/Out jacks. I have to admit i have sometimes small problems to understand documents and i need hardware to work with.
Well, a quick workaround for "getting" quickly a midi device, would be to set an Android phone as an usb midi device when you plug the phone to a host device, for example windows PC. Then i guess you could use any midi keyboard app, (there are some on the playstore) to make the notes or codes going out from the device.
I would be more than happy to collaborate as i am very interested to make my own project successful, bur as you said, there are many things to be understood, not only midi protocol, but the usb along midi as well.
I have seen similar Arduino shield projects but im interested to implement it on the esp32-s3.
I know you are busy as i have seen you in different esp32 forums, but would be nice to get you involved as i can see you have deep knowledge on this platform
This is what i do for living for few years, i am esp32 programming freelancer.
Yes, ive been using android smartphone to implement simple midi code for ble midi on esp32.
This is what i do for living for few years, i am esp32 programming freelancer.
Yes, ive been using android smartphone to implement simple midi code for ble midi on esp32.
Could you share the link for that project? Maybe i can get some ideas and try to implement them with my usb midi device. Thanks again for your replies! Highly appreciated
Thanks for the feedback. I have a side question. Well, first, if this is not the right place to ask, i would like to know where would it be bit if you want to clarify, it will be welcome. I have an adafruit esp32-s3 feather board with only one usb type c board. My goal is still the one i commented in the begining, to convert a usb-midi controller to Bluetooth. But i am confused with the differences between the board i have and the dev kit versions of the s3. I mean, s3 boards are capable to handle natively usb, nevertheless, the s3 dev kit has d+ and d- pins that are used for data lanes. The feather version does not have them. I also want to power the external device via the USB port(i guess my midi device requires at least 5v and 500 ma) so i guess the 3.7v battery will be able to provide power to the board and the usb midi device i want to use. So, the question now is, does my board can really do the job via the built in usb c port or does it needs a breakout? And, does the dev kit version needs a breakout anyway? I am planning to buy the next board anyway: https://wiki.banana-pi.org/BPI-Leaf-S3
The espresiff version has 2 ports, while the banana pi has only one with the same pin layout. I am willing to add a usb breakout either to my feather board or to the banana leaf, but its still not clear the situation with the usb otg (host) funtion, as I want to read the external midi messages from my pedalboard.
Thanks for your time and clarifications in advance
@touchgadget @csBlueChip
Did you manage to make the USB host to receive MIDI? Im planning to use a esp32-s3 with an additional usb breakout to make an external controller wireless via Bluetooth. Let me know your progress. Thanks
@josehdx Yes, see the following for hex dumps of received MIDI 4 byte records. The code is in the examples directory. Note this is a different repo from this one.
https://github.com/touchgadget/esp32-usb-host-demos#usbhmidi----usb-host-midi
For BLE MIDI I would take a look at github. For example, see https://github.com/max22-/ESP32-BLE-MIDI There may be other repos.
I mean, s3 boards are capable to handle natively usb, nevertheless, the s3 dev kit has d+ and d- pins that are used for data lanes. The feather version does not have them.
ESP32-S3 boards with a single USB connector usually route d+ and d- signals go to the USB connector so the signals are not available on board pins. To connect a MIDI device use a USB type C or micro to USB OTG host cable. But this only solves the problem of connecting GND, D+, and D-. If the MIDI device has its power supply adapter, it may fine without USB 5V. If it does not, you may have to consult the board vendor schematics or forum on how best to connect 5V so it goes out the USB connector. This might mean cutting traces and/or removing blocking diodes. Loading programs will be tricky because it probably requires removing the USB OTG cable and disconnecting the 5V power source.
@chegewara
Thank you , and again thank you so much for your answer. I wasn't able to find this answer even on the espresiff forum. In that scenario, it will be better to have a dev kit as it has the 5v pins available, i just need a breakout, and will be ready to go I guess.
Thank you again!