avrdude icon indicating copy to clipboard operation
avrdude copied to clipboard

Differentiate multiple USB programmers of the same VID/PID (libusb or hidapi or libftdi)

Open mcuee opened this issue 3 years ago • 4 comments

Ref: the following enhancement requests talk about usbasp

  • https://github.com/avrdudes/avrdude/issues/619
  • https://github.com/avrdudes/avrdude/issues/718

But this can be extended to other USB based programmers. If you have multiple programmers of the same types connected, the best is to differentiate them by serial number. Or you can differentiate with the Manufacturer/Product strings. Then there is another possibility is to use the locations (USB ports).

mcuee avatar May 22 '22 10:05 mcuee

Specific to hidapi: at least you can get the vendor_id, product_id, path, serial_number. Ref: https://github.com/libusb/hidapi/blob/master/hidtest/test.c

mcuee avatar May 22 '22 10:05 mcuee

For libusb, same that you can get the vendor_id, product_id and serial_number. You can also get the path using libusb_get_bus_number and libusb_get_port_number. https://github.com/libusb/libusb/blob/master/examples/xusb.c

mcuee avatar May 22 '22 11:05 mcuee

For libftdi, ftdi_usb_get_strings() can probably be used. https://www.intra2net.com/en/developer/libftdi/documentation/ftdi_8h.html#a17e8eae09e58bd3bfeeab8decfce7fd9

mcuee avatar May 22 '22 11:05 mcuee

Ref: OpenOCD libusb-1.0 helper may give some hints as well. Ref: https://github.com/openocd-org/openocd/blob/master/src/jtag/drivers/libusb_helper.c

It can be done using libusb-0.1 API as well since avrdude is still mostly using libusb-0.1 API.

mcuee avatar Jun 12 '22 09:06 mcuee

I have some working code for USBASP, specifying programmer with -P usb:xxx:xxx.

Any interest in adding this to the project?

MikeRaffa avatar Mar 08 '23 19:03 MikeRaffa

How did you solve the multiplatform aspects? (Windows, Linux, MacOS, BSD)

dl8dtl avatar Mar 08 '23 19:03 dl8dtl

Any interest in adding this to the project?

It would be great if you could create a PR for this! usb_libusb.c and usb_hidapi.c already reads the serial number from the device and store this in a string, usbsn, but it looks like usbasp.c uses libusb directly.

MCUdude avatar Mar 08 '23 19:03 MCUdude

As a remark (since I once implemented the serno matcher in stk500v2/jtag): serial numbers can sometimes be long, and it makes most sense to just match the trailing part. (OpenOCD requires the full serial number, which is hard to type when typing manually on the commandline.)

dl8dtl avatar Mar 08 '23 20:03 dl8dtl

Sounds like there's some value to potentially adding that. I'll do some cleanup/testing and then submit something if it looks like it's working.

I've only tested on Linux so far. I could check on Windows/Mac as well; not sure what the usblib differences are there. The method I'm using is by USB bus (usb:xxx:xxx). Not sure if serial no. is prefereable.

MikeRaffa avatar Mar 08 '23 20:03 MikeRaffa

Using a serial number is much preferable – however, I'm afraid most (all?) USBasp implementations just don't offer a serial number at all. For devices lacking a serial number, it might make some sense to offer alternative ways to distinguish different programmers. But then, I'm afraid every operating system needs to be treated differently. Either case, please don't forget to update the documentation, or at least to provide a text block we can include into the documentation files. (We can handle the respective markup if you are unfamiliar with that.)

dl8dtl avatar Mar 08 '23 20:03 dl8dtl

For devices lacking a serial number, it might make some sense to offer alternative ways to distinguish different programmers. But then, I'm afraid every operating system needs to be treated differently.

The USB bus:port number may be another way to distinguish the devices: libusb_get_port_number() and libusb_get_port_numbers(). But then libusb-1.0 API is needed. https://libusb.sourceforge.io/api-1.0/group__libusb__dev.html#ga14879a0ea7daccdcddb68852d86c00c4 https://github.com/libusb/libusb/blob/master/examples/listdevs.c

mcuee avatar Mar 11 '23 14:03 mcuee

Using a serial number is much preferable – however, I'm afraid most (all?) USBasp implementations just don't offer a serial number at all.

Indeed, unless the user is willing to rebuild the FW and program different units with different serial number.

It would be good to such FW capability to usbasp FW but then we need to have a host software as well (Microchip PICKit 2 FW and host SW have the capability to program the SN and differentiate the units).

@dioannidis I am not so sure if you are interested to add such capability to your usbasp FW. The HID side-channel may be used to program the serial number.

mcuee avatar Mar 11 '23 15:03 mcuee

@mcuee

@dioannidis I am not so sure if you are interested to add such capability to your usbasp FW. The HID side-channel may be used to program the serial number.

I already implement it. The serial number ( 4 digits only ) is stored in eeprom and one can read / write it via HID calls only ( do I need to support control requests calls also ? ) . Didn't commit it yet cause I didn't test it enough.

FYI, I choose to add, the eeprom read / write functionality, to the V-USB driver ( see https://github.com/obdev/v-usb/pull/35 ) instead of using the usbFunctionDescriptor / usbFunctionSetup mechanism, because I thought that one may choose to store all the descriptors to eeprom instead of flash to save space.

I'll try to find time to test it little more and upload it ASAP .

dioannidis avatar Mar 11 '23 16:03 dioannidis

@mcuee, @MCUdude , @dl8dtl

Also we need to inform the avrdude that there is a serial number descriptor read / write functionality using the 4 bytes capability control request response ( USBASP_FUNC_GETCAPABILITIES ).

Currently, the capability response ( in my firmware ) uses the 1st byte to report if it supports TPI and / or HIDUART and the 2nd byte if the USBasp device uses a different crystal from the default 12MHz.

<snip>
/* USBASP capabilities */
#define USBASP_CAP_0_TPI                0x01
#define USBASP_CAP_6_UART               0x40
#define USBASP_CAP_HIDUART              0x80

#define USBASP_CAP_12MHZ_CLOCK          0x00
#define USBASP_CAP_16MHZ_CLOCK          0x01
#define USBASP_CAP_18MHZ_CLOCK          0x02
#define USBASP_CAP_20MHZ_CLOCK          0x03
<snip>

Where do you propose to add the serial number capability ?

<snip>
#define USBASP_FUNC_SETSERIALNUMBER	125
#define USBASP_FUNC_GETSERIALNUMBER	126
<snip>

Do I need to change the USBASP_FUNC_SETSERIALNUMBER, USBASP_FUNC_GETSERIALNUMBER defines to other values ?

dioannidis avatar Mar 11 '23 17:03 dioannidis

@mcuee, @MCUdude , @dl8dtl

Also we need to inform the avrdude that there is a serial number descriptor read / write functionality using the 4 bytes capability control request response ( USBASP_FUNC_GETCAPABILITIES ).

@dioannidis

I am not so sure if avrdude wants to be involved in that function of write the serial number as it is specific to your usbasp firmware. I tend to think you have to provide that functionality through your host application, outside of avrdude.

avrdude will just read the serail number of your device and utilize to differentiate different units.

Adding @stefanrueger as well in this discussion.

mcuee avatar Mar 12 '23 02:03 mcuee

Releavent discussion from libusb project. https://github.com/libusb/libusb/pull/1258

Take note HIDAPI provides multiple aspects of the HID device for matching the device.

Device Found
  type: 046d c52b
  path: \\?\HID#VID_046D&PID_C52B&MI_00#7&34f0fd76&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}\KBD
  serial_number:
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      1203
  Interface:    0
  Usage (page): 0x6 (0x1)
  Bus type: 1

mcuee avatar Mar 12 '23 12:03 mcuee

I am not so sure if avrdude wants to be involved in that function of write the serial number as it is specific to your usbasp firmware. I tend to think you have to provide that functionality through your host application, outside of avrdude.

Me neither.

Since it's HIDAPI, I guess it would be easy enough to offer some Python implementation, and thus be quite platform independant. If I were you @dioannidis I wouldn't bother too much with capabilties: just hack a Python script that produces a semi-random serial number, and fire it to the USBasp device: if the device can handle it, it's fine. If not, the user is just at the same point where it has been before. ;-)

\?\HID#VID_046D&PID_C52B&MI_00#7&34f0fd76&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}\KBD

Pah, that would be really lengthy to type …

dl8dtl avatar Mar 12 '23 21:03 dl8dtl

@dl8dtl, @mcuee , @MCUdude , @stefanrueger

Well, I tend to think that USBasp goes hand to hand with avrdude, over 90% ( or more ) of the time. I don't think I ever used a USBasp device using other software which is not using avrdude under the hood .

As I already wrote above, I've already implemented the functionality in my USBasp fork locally and after some tests this week it will be commited .

Anyway, I thought that it will be useful if the avrdude project "dictates" / "proposes" an entry point for all the usbasp firmwares to implement the serial number change set, HIDAPI or not. That will solve the platform independent issue also ...

dioannidis avatar Mar 12 '23 21:03 dioannidis

I've looked into this a bit more and come to a couple of conclusions:

1.) The USB path method for differentiating between multiple USBASP devices isn't great for my workflow. At least on my Ubuntu VM, the -P usb:xxx:yyy doesn't correspond to physical hub/port, so it changes each bootup or when a device is plugged in. That means it's only useful within a single session and requires some experimentation to determine which USBASP is which.

2.) I reprogrammed the USBASP firmware with the code from here https://www.fischl.de/usbasp/ with a specified serial number in the firmware/usbconfig.h file, and it seemed like a relatively painless process. It does require two USBASPs (a target and another to program the firmware on the target). That's now working with my avrdude modification to allow for -P usb:xxx (where xxx is serial number).

I still have the option of specifying -P usb:xxx:yyy for the USB bus/device in my changes here, and it seems to make sense to leave that in as well for users who don't want to go to the trouble of reprogramming their USBASP. Any thoughts?

MikeRaffa avatar Mar 13 '23 06:03 MikeRaffa

@MikeRaffa

Where can I found your avrdude patch reg. serial number ? I see no repos at https://github.com/MikeRaffa ...

dioannidis avatar Mar 13 '23 11:03 dioannidis

I haven't uploaded anything yet. Just got it working in my environment. Still need to get it working with the different libusb versions and test on Windows.

MikeRaffa avatar Mar 13 '23 21:03 MikeRaffa

Well, there is no obligation to have everything already tested and final before uploading it to your own forked repository. ;-) So if you decide to publish your own fork, others just get the possibility to peek at your work already.

dl8dtl avatar Mar 14 '23 08:03 dl8dtl

@mcuee

I uploaded a development version of the firmware ( for atmega88 ) at https://www.nephelae.eu/usbasp88.zip . Also the host app ( only for win64 ) is at https://www.nephelae.eu/vusbserialnumber-x86_64-win64.zip .

Any feedback is appreciated ...

C:\VUSBSerialNumber.exe -h

USBasp Serial Number utility (for FW https://github.com/dioannidis/usbasp )

 -l  List USBasp devices with HID support.
 -s  Serial Number to update ( 4 Digits numeric only i.e. 3456, 2222, etc ).
 -i  USBasp list index. Used with -s if there are more than one USBasp connected at the same time.

examples

Update the first found USBasp's serial number
 VUSBSerialNumber -s 1234

Update the specific USBasp's serial number using USBasp list index
 VUSBSerialNumber -i 2 -s 1234

dioannidis avatar Mar 14 '23 19:03 dioannidis

Preliminary version of -P bus/device and serial_number support for USBASP is here https://github.com/MikeRaffa/avrdude.

MikeRaffa avatar Mar 15 '23 06:03 MikeRaffa

@dioannidis

Can you provide the hex file for ATmega8A? My USBAsp unit based on ATmega88 is a bit fragile and not as robust as the ones based on ATmega8A. Thanks.

mcuee avatar Mar 15 '23 07:03 mcuee

@mcuee

Can you provide the hex file for ATmega8A? My USBAsp unit based on ATmega88 is a bit fragile and not as robust as the ones based on ATmega8A. Thanks.

Of course. Here it is https://www.nephelae.eu/usbasp8.zip .

dioannidis avatar Mar 15 '23 08:03 dioannidis

@mcuee

Please download the hex files again, because I forgot to build them for 12Mhz crystal . I have a custom usbasp board which I test various crystals and it had an 18MHz on .... Apologies ...

( don't forget this firmware implements a composite device so only the mingw build of avrdude will work .... )

dioannidis avatar Mar 15 '23 09:03 dioannidis

@dioannidis

Your FW works very well. Itested with two USBASP (Atmega8A) units.

The only minor issue is with the host software -- I need to hit a return to exit. Edit: another issue: somehow the serial number 6666 becomes 0666.

PS C:\work\avr\avrdude_test\avrdude_bin\usbasp_sn> .\VUSBSerialNumber.exe -l

USBasp Serial Number utility (for FW https://github.com/dioannidis/usbasp )


USBasp List Index: 0

type: 16C0 05DC
path: \\?\HID#VID_16C0&PID_05DC&MI_01#7&259174a7&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
serial number: 0000
Manufacturer:  www.fischl.de
Product:       USBasp
Release:       1.10
Interface:     1

USBasp List Index: 1

type: 16C0 05DC
path: \\?\HID#VID_16C0&PID_05DC&MI_02#7&1ba36e5&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
serial number: 0000
Manufacturer:  www.fischl.de
Product:       USBasp
Release:       1.10
Interface:     2

USBasp List Index: 2

type: 16C0 05DC
path: \\?\HID#VID_16C0&PID_05DC&MI_01#7&28dd995c&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
serial number: 0888
Manufacturer:  www.fischl.de
Product:       USBasp
Release:       1.10
Interface:     1

USBasp List Index: 3

type: 16C0 05DC
path: \\?\HID#VID_16C0&PID_05DC&MI_02#7&5065b9a&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
serial number: 0888
Manufacturer:  www.fischl.de
Product:       USBasp
Release:       1.10
Interface:     2


PS C:\work\avr\avrdude_test\avrdude_bin\usbasp_sn> .\VUSBSerialNumber.exe -i 0 -s 6666

USBasp Serial Number utility (for FW https://github.com/dioannidis/usbasp )


USBasp List Index: 0

type: 16C0 05DC
path: \\?\HID#VID_16C0&PID_05DC&MI_01#7&259174a7&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
serial number: 0000
Manufacturer:  www.fischl.de
Product:       USBasp
Release:       1.10
Interface:     1

Updating USBasp Serial Number ...

USBasp List Index: 0

type: 16C0 05DC
path: \\?\HID#VID_16C0&PID_05DC&MI_01#7&259174a7&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
serial number: 0666
Manufacturer:  www.fischl.de
Product:       USBasp
Release:       1.10
Interface:     1

mcuee avatar Mar 15 '23 11:03 mcuee

@MikeRaffa Your fork is good for Windows as well using serial number. I connected two USBasp units (one with serial number 0001 and the other one with serial number 0888) , and then another USBasp as target to the unit with serial number 0888. git main will fail but your patch works fine.

Take note all three units are still using WinUSB for Interface 0 and HID driver for Interface 1 (USB composite device) so I have to use MinGW to build avrdude binary due to issue #968 as mentioned @dioannidis.

  • #968
PS C:\work\avr\avrdude_test\avrdude_bin\usbasp_sn> .\VUSBSerialNumber.exe -l

USBasp Serial Number utility (for FW https://github.com/dioannidis/usbasp )


USBasp List Index: 0

type: 16C0 05DC
path: \\?\HID#VID_16C0&PID_05DC&MI_01#8&6001234&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
serial number: 0001
Manufacturer:  www.fischl.de
Product:       USBasp
Release:       1.10
Interface:     1

USBasp List Index: 1

type: 16C0 05DC
path: \\?\HID#VID_16C0&PID_05DC&MI_01#7&28dd995c&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
serial number: 0888
Manufacturer:  www.fischl.de
Product:       USBasp
Release:       1.10
Interface:     1

USBasp List Index: 2

type: 16C0 05DC
path: \\?\HID#VID_16C0&PID_05DC&MI_02#7&5065b9a&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
serial number: 0888
Manufacturer:  www.fischl.de
Product:       USBasp
Release:       1.10
Interface:     2

PS C:\work\avr\avrdude_test\avrdude_bin\usbasp_sn> .\avrdude -c usbasp -p m8a

avrdude error: program enable: target does not answer (0x01)
avrdude error: initialization failed, rc=-1
        - double check the connections and try again
        - use -B to set lower ISP clock frequency, e.g. -B 125kHz
        - use -F to override this check


avrdude done.  Thank you.

PS C:\work\avr\avrdude_test\avrdude_bin\usbasp_sn> .\avrdude_issue973 -c usbasp -p m8a -P usb:0888

avrdude_issue973: AVR device initialized and ready to accept instructions
avrdude_issue973: device signature = 0x1e9307 (probably m8a)

avrdude_issue973 done.  Thank you.

mcuee avatar Mar 15 '23 12:03 mcuee

@mcuee

The only minor issue is with the host software -- I need to hit a return to exit. Edit: another issue: somehow the serial number 6666 becomes 0666.

Fixed both issues . Please download again the host app https://www.nephelae.eu/vusbserialnumber-x86_64-win64.zip .

dioannidis avatar Mar 15 '23 15:03 dioannidis