Make libusb_device_handle reference counted

[ghost]

Since opening a device multiple times causes issues (depending on OS), an application needs to manage a single libusb_device_handle for all tasks, even if it's some sort of multi-function device, which can handled by multiple independent modules.

I'm arguing that making libusb_device_handle refcounted would make this easier, simplify application logic, and free libusb users from either having to refcount it themselves, or having to worry about opening a device multiple times. usb_close() would simply decrease the refcount by 1.

It may also make usb_close() simpler if there are still "in flight" transfers (but not sure about libusb internals).

Somewhat related to what I wrote in #700.

[dickens]

If I understand correctly, is the proposal to have a single libusb_device_handle per device that gets created on the first call to libusb_open() with subsequent calls to libusb_open() returning the same handle but with an increased reference count?

[ghost]

Hm, not necessarily. But maybe that would work too. I think you can split this into 3 different possible things (that are orthogonal):

1. A refcount for user convenience (user can call functions to ref/unref a device)
2. Like 1., but libusb uses it for in-flight transfers, so that ongoing transfers keep the device alive with a refcount until finished
3. making libusb_open() return the same handle with an increased refcount if already open

[dickens]

I was hoping that what I wrote was not your proposal, and it doesn't sound like it is.

I'm all for approach 2--I think it is a good enhancement.

[minus7]

That would also help for closing a device handle in a failed transfer callback, since currently calling libusb_close() in a transfer callback deadlocks.

[mcuee]

Repost from #700 as the issue was closed by the unknown poster.

---

Assume you have a multi-threaded application that has a lot of in-flight async. transfers (such as continuous data transfer from a device). If you want to safely close the USB device, you need to:

1. keep a list of all transfers
2. call libusb_cancel_transfer() on each of it
3. wait until their completion callbacks have been called
4. call libusb_close

I think this is too complex. I'd really just want to be able to stop transfers and/or close a device in a non-blocking and instant way. I'd prefer something like this:

1. being able to instanly stop libusb_transfers and make libusb completely forget about them
2. being able to just close libusb_device_handles without having to worry about ongoing transfers
3. maybe provide an event queue to signal libusb_transfer completion instead of a tricky callback

Now, the first suggestion would probably be to set the LIBUSB_TRANSFER_FREE_BUFFER and LIBUSB_TRANSFER_FREE_TRANSFER flags on each transfer. But you'd still need to clear the completion callbacks (see below), and it doesn't seem to solve this problem anyway (other than fixing memory leaks).

libusb_cancel_transfer() is not enough either. It will still call the completion callback. The function doesn't even tell you whether a callback will be called. (It can return LIBUSB_ERROR_NOT_FOUND when the transfer was already canceled, and the completion callback is still to be called.)

Can you safely set libusb_transfer.callback to NULL? The user has no control over when libusb reads this field. Do you need to lock via libusb_lock_events()? Do you need to do it on whichever thread you call libusb_handle_events()? Or is it not possible? In general, I'm rather confused about in which context completion callbacks can be called.

And with all that, you still need to know when you can call libusb_close(). If you just "orphan" all transfers (like described above, and for the sake of your application logic), you don't know when the transfers actually complete, so you can close the device.

Can you just call libusb_close() with transfers still ongoing? It seems it somehow synchronously cancels transfers, prints an ugly warning, and calls the completion callbacks synchronously. But judging from past issues, this is more like a hack to make API user errors less fatal.

Note that this doesn't just affect closing devices. If it's some sort of multi-function device, the application may have multiple subsystems (using different USB endpoints) that you may want to start/stop independently without closing the device. So if you want to stop transfers instantly for simplicity, you're still... troubled.

It's also possible that I'm overthinking this, and there's a way to handle this correctly. I've spent a lot time thinking about this though, and I've wrote my own 500 line helper which "garbage collects" libusb_transfers (and optionally libusb_device_handles) once nothing needs/wants them anymore. Judging by how many vaguely similar issues there are on this repo (where similar things were discussed), I assume this is a common problem though.