eblot
Cannot close Controller associated with disconnected device
Hi,
When calling close (or terminate for I2cController) on an instance associated with a device that have been physically disconnect the following exceptions are thrown:
Traceback (most recent call last):
File "/code/venv/lib/python3.8/site-packages/pyftdi/ftdi.py", line 2044, in _ctrl_transfer_out
return self._usb_dev.ctrl_transfer(
File "/code/venv/lib/python3.8/site-packages/usb/core.py", line 1036, in ctrl_transfer
ret = self._ctx.backend.ctrl_transfer(
File "/code/venv/lib/python3.8/site-packages/usb/backend/libusb1.py", line 875, in ctrl_transfer
ret = _check(self.lib.libusb_control_transfer(
File "/code/venv/lib/python3.8/site-packages/usb/backend/libusb1.py", line 595, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 19] No such device (it may have been disconnected)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./bin/ft232h-reconnect-test.py", line 17, in <module>
controller.close()
File "/code/venv/lib/python3.8/site-packages/pyftdi/gpio.py", line 103, in close
self._ftdi.close()
File "/code/venv/lib/python3.8/site-packages/pyftdi/ftdi.py", line 592, in close
self.set_bitmode(0, Ftdi.BitMode.RESET)
File "/code/venv/lib/python3.8/site-packages/pyftdi/ftdi.py", line 1229, in set_bitmode
if self._ctrl_transfer_out(Ftdi.SIO_REQ_SET_BITMODE, value):
File "/code/venv/lib/python3.8/site-packages/pyftdi/ftdi.py", line 2048, in _ctrl_transfer_out
raise FtdiError('UsbError: %s' % str(ex))
pyftdi.ftdi.FtdiError: UsbError: [Errno 19] No such device (it may have been disconnected)
Test code used (tested with pyftdi 0.50.2 and an Adafruit FT232H):
from pyftdi.gpio import GpioMpsseController
print('Creating, configuring, writing and closing a GpioMpsseController...')
controller = GpioMpsseController()
controller.configure(
'ftdi://ftdi:ft232h/1',
frequency=100,
direction=0xFFFF,
initial=0x0000)
input('Now, disconnect and reconnect hardware and press enter.')
controller.close()
It would be useful with some way to clean up the controller even for disconnected devices. As a workaround I have done the following:
UsbTools.release_device(controller._ftdi._usb_dev)
controller.close()
UsbTools.flush_cache()
controller.configure(...)
which isn't the nicest because I need to access private fields. This works though. My use case is I have a Adafruit FT232H connected to my laptop docking station at work and have a service running on the laptop that controls my standing desk. When I undock/redock my laptop I want the service to automatically rediscover the USB device and resume controlling the desk.
Would a force=True argument to the close() method make sense? Other ideas on how to achieve my use case? I might have misunderstood the entire thing :).
I guess this could definitely be improved, I never tested this use case.
I'll try to have a look as soon as I get some spare time.
The release_interface call in Ftdi.close() may also raise an USBError. It prevents close() from cleaning up properly. Stack trace:
Traceback (most recent call last):
File "./bin/ft232h-reconnect-test.py", line 21, in <module>
controller.close()
File "./venv/lib/python3.8/site-packages/pyftdi/gpio.py", line 103, in close
self._ftdi.close()
File "./venv/lib/python3.8/site-packages/pyftdi/ftdi.py", line 601, in close
release_interface(dev, self._index - 1)
File "./venv/lib/python3.8/site-packages/usb/util.py", line 217, in release_interface
device._ctx.managed_release_interface(device, interface)
File "./venv/lib/python3.8/site-packages/usb/core.py", line 102, in wrapper
return f(self, *args, **kwargs)
File "./venv/lib/python3.8/site-packages/usb/core.py", line 182, in managed_release_interface
self.backend.release_interface(self.handle, i)
File "./venv/lib/python3.8/site-packages/usb/backend/libusb1.py", line 815, in release_interface
_check(self.lib.libusb_release_interface(dev_handle.handle, intf))
File "./venv/lib/python3.8/site-packages/usb/backend/libusb1.py", line 595, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 19] No such device (it may have been disconnected)
I have not seen this behavior on macOS, may be it is a Linux thing, i’ll try on Linux then.
Weird, I do not see any error on Linux Mint:
eblot@vspey:~/Sources/Git/pyftdi$ PYTHONPATH=. pyftdi/tests/ftdi.py
s.Please disconnect FTDI device
FTDI device may be gone: UsbError: [Errno 32] Pipe error
Please reconnect FTDI device
FTDI device not initialized
FTDI device not initialized
FTDI device detected
.
----------------------------------------------------------------------
Ran 3 tests in 10.265s
Can you run this test on your host? Thanks.
The test works on my machine:
> PYTHONPATH=. pyftdi/tests/ftdi.py
s.Please disconnect FTDI device
FTDI device may be gone: UsbError: [Errno 19] No such device (it may have been disconnected)
Please reconnect FTDI device
FTDI device detected
.
----------------------------------------------------------------------
Ran 3 tests in 9.579s
OK (skipped=1)
Though, my test script (with pyftdi 0.51.2) still raises from within release_interface. Maybe the behavior in libusb is different when using MPSSE?
I use the script in first post, you can also modify the test case like this:
diff --git a/pyftdi/tests/ftdi.py b/pyftdi/tests/ftdi.py
index 096f113..bf4f880 100755
--- a/pyftdi/tests/ftdi.py
+++ b/pyftdi/tests/ftdi.py
@@ -137,7 +137,7 @@ class DisconnectTestCase(TestCase):
log = logging.getLogger('pyftdi.tests.ftdi')
url = environ.get('FTDI_DEVICE', 'ftdi:///1')
ftdi = Ftdi()
- ftdi.open_from_url(url)
+ ftdi.open_mpsse_from_url(url, direction=0xFFFF, frequency=1e6)
self.assertTrue(ftdi.is_connected, 'Unable to connect to FTDI')
print('Please disconnect FTDI device')
while ftdi.is_connected:
@@ -151,7 +151,7 @@ class DisconnectTestCase(TestCase):
while True:
UsbTools.flush_cache()
try:
- ftdi.open_from_url(url)
+ ftdi.open_mpsse_from_url(url, direction=0xFFFF, frequency=1e6)
except (FtdiError, UsbToolsError):
log.debug('FTDI device not detected')
sleep(0.1)
With these changes I get:
> PYTHONPATH=. pyftdi/tests/ftdi.py
s.Please disconnect FTDI device
FTDI device may be gone: UsbError: [Errno 19] No such device (it may have been disconnected)
E
======================================================================
ERROR: test_close_on_disconnect (__main__.DisconnectTestCase)
Validate close after disconnect.
----------------------------------------------------------------------
Traceback (most recent call last):
File "pyftdi/tests/ftdi.py", line 149, in test_close_on_disconnect
ftdi.close()
File "/home/daniel/code/pyftdi/pyftdi/ftdi.py", line 601, in close
release_interface(dev, self._index - 1)
File "/home/daniel/code/pyftdi/venv/lib/python3.8/site-packages/usb/util.py", line 217, in release_interface
device._ctx.managed_release_interface(device, interface)
File "/home/daniel/code/pyftdi/venv/lib/python3.8/site-packages/usb/core.py", line 102, in wrapper
return f(self, *args, **kwargs)
File "/home/daniel/code/pyftdi/venv/lib/python3.8/site-packages/usb/core.py", line 182, in managed_release_interface
self.backend.release_interface(self.handle, i)
File "/home/daniel/code/pyftdi/venv/lib/python3.8/site-packages/usb/backend/libusb1.py", line 815, in release_interface
_check(self.lib.libusb_release_interface(dev_handle.handle, intf))
File "/home/daniel/code/pyftdi/venv/lib/python3.8/site-packages/usb/backend/libusb1.py", line 595, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 19] No such device (it may have been disconnected)
----------------------------------------------------------------------
Ran 3 tests in 3.377s
FAILED (errors=1, skipped=1)
Weird, I still cannot reproduce it with your patch, it generates several different error messages, but the error is always handled:
$ PYTHONPATH=. pyftdi/tests/ftdi.py
s.Please disconnect FTDI device
FTDI device may be gone: UsbError: [Errno None] Other error
Please reconnect FTDI device
FTDI device not initialized
FTDI device detected
.
----------------------------------------------------------------------
Ran 3 tests in 12.966s
OK (skipped=1)
$ PYTHONPATH=. pyftdi/tests/ftdi.py
s.Please disconnect FTDI device
FTDI device may be gone: UsbError: [Errno 32] Pipe error
Please reconnect FTDI device
FTDI device not initialized
FTDI device detected
.
----------------------------------------------------------------------
Ran 3 tests in 19.519s
OK (skipped=1)
$ PYTHONPATH=. pyftdi/tests/ftdi.py
s.Please disconnect FTDI device
FTDI device may be gone: UsbError: [Errno 5] Input/Output Error
Please reconnect FTDI device
FTDI device not initialized
FTDI device detected
.
----------------------------------------------------------------------
Ran 3 tests in 19.519s
OK (skipped=1)
I will try on Linux with your patch.
Same behaviour on Linux Mint:
s.Please disconnect FTDI device
FTDI device may be gone: UsbError: [Errno 32] Pipe error
Please reconnect FTDI device
FTDI device not initialized
FTDI device not initialized
FTDI device not initialized
FTDI device not initialized
FTDI device not initialized
FTDI device detected
.
----------------------------------------------------------------------
Ran 3 tests in 7.142s
OK (skipped=1)
Weird I checked the docs for libusb_release_interface[1] and it states that the error LIBUSB_ERROR_NO_DEVICE can be returned which is what's happening for me. In the python usb library is turned into an USBError exception [2]. From the docs and code one can't assume that release_interface will not raise an exception. I don't understand why this is not happening for you though. Is it different libusb/pyusb versions? I have these:
libusb 1.0.23
pyusb 1.0.2
Other than that I don't know... I still have to call UsbTools.release_device manually from my code to be able to reconfigure the controller after the device have been reconnected.
1: http://libusb.sourceforge.net/api-1.0/group__libusb__dev.html#ga49b5cb0d894f6807cd1693ef29aecbfa 2: https://github.com/pyusb/pyusb/blob/v1.0.2/usb/backend/libusb1.py#L595
I’ll have another look at the UsbTools release. I’d really like to reproduce this somehow.
The release_interface call in Ftdi.close() may also raise an USBError.
I'm also seeing that error with pyftdi 0.52.0 on Ubuntu 20 using Python 3.7:
Exception in thread usb-monitor:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "hardware_ctl.py", line 93, in _usb_monitor
i2c.configure("ftdi:///1")
File "/home/thijs/.virtualenvs/foo/lib/python3.7/site-packages/pyftdi/i2c.py", line 517, in configure
frequency = self._ftdi.open_mpsse_from_url(url, **kwargs)
File "/home/thijs/.virtualenvs/foo/lib/python3.7/site-packages/pyftdi/ftdi.py", line 645, in open_mpsse_from_url
devdesc, interface = self.get_identifiers(url)
File "/home/thijs/.virtualenvs/foo/lib/python3.7/site-packages/pyftdi/ftdi.py", line 421, in get_identifiers
cls.DEFAULT_VENDOR)
File "/home/thijs/.virtualenvs/foo/lib/python3.7/site-packages/pyftdi/usbtools.py", line 333, in parse_url
default_vendor)
File "/home/thijs/.virtualenvs/foo/lib/python3.7/site-packages/pyftdi/usbtools.py", line 437, in enumerate_candidates
devices = cls.find_all(vps)
File "/home/thijs/.virtualenvs/foo/lib/python3.7/site-packages/pyftdi/usbtools.py", line 121, in find_all
description = UsbTools.get_string(dev, dev.iProduct)
File "/home/thijs/.virtualenvs/foo/lib/python3.7/site-packages/pyftdi/usbtools.py", line 575, in get_string
return usb_get_string(device, stridx)
File "/home/thijs/.virtualenvs/foo/lib/python3.7/site-packages/usb/util.py", line 322, in get_string
langid
File "/home/thijs/.virtualenvs/foo/lib/python3.7/site-packages/usb/control.py", line 175, in get_descriptor
data_or_wLength = desc_size)
File "/home/thijs/.virtualenvs/foo/lib/python3.7/site-packages/usb/core.py", line 1077, in ctrl_transfer
self.__get_timeout(timeout))
File "/home/thijs/.virtualenvs/foo/lib/python3.7/site-packages/usb/backend/libusb1.py", line 901, in ctrl_transfer
timeout))
File "/home/thijs/.virtualenvs/foo/lib/python3.7/site-packages/usb/backend/libusb1.py", line 604, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 19] No such device (it may have been disconnected)
also seeing this after a close and physical disconnect and then reconnect and attempt to open ftdi.
using pyftdi 0.55.4, raspberry os 12, and python 3.11
