Conflicting bleak library version with HomeAssistant 2022.8

[carsonb]

It looks like 2022.8 has updated to bleak 0.15.0 https://github.com/home-assistant/core/pull/75941

When trying to use 4.0.0 of sensor.airthings_wave I get the following error in the logs:

Unable to install package bleak==0.14.3: ERROR: Cannot install bleak==0.14.3 because these package versions have conflicting dependencies. ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

[ismarslomic]

Just upgraded HA to v2022.8.0 with already installed sensor.airthings_wave v4.0.0 and got same error statement in HA logs.

[lymanepp]

HA 2022.8 added native bluetooth support using the bleak library - https://www.home-assistant.io/blog/2022/08/03/release-20228/#first-class-bluetooth-support

I modified the manifest.json file to reference bleak 0.15.0, but it's still not working. Here's what the log shows.

2022-08-03 14:33:47.636 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.airthings_wave
2022-08-03 14:33:57.661 INFO (SyncWorker_5) [custom_components.airthings_wave.sensor] Found 1 airthings device(s)
2022-08-03 14:33:57.730 WARNING (MainThread) [homeassistant.components.sensor] Setup of sensor platform airthings_wave is taking over 10 seconds.
2022-08-03 14:33:57.749 INFO (SyncWorker_5) [custom_components.airthings_wave.airthings] Not able to connect to 04:EE:03:BB:B4:13/
2022-08-03 14:33:57.749 ERROR (SyncWorker_5) [custom_components.airthings_wave.airthings] Not getting device info because failed to connect to device.
2022-08-03 14:33:57.793 INFO (SyncWorker_5) [custom_components.airthings_wave.airthings] Not able to connect to 04:EE:03:BB:B4:13

[kongjudas]

Just a question: As the this integration is unsupported for the time being, will devs focus on the new BT integration, if even possible?

[lone-cloud]

bleak v0.15 is considered a potentially breaking release based on semserv versioning of a 0.x pre-release (aka unstable). I'm not a python dev, but it looks like poor debugging output as we don't actually know what the exception is here: https://github.com/custom-components/sensor.airthings_wave/blob/master/custom_components/airthings_wave/airthings.py#L245 I believe that anyone with the faintest knowledge of python should be able to easily solve this. If not, I'll work on this tomorrow, as I'd like to be on the latest release of HA with this integration working.

[lymanepp]

I added logging of the exception details:

This might be the root cause of the issue: got Future <Future pending> attached to a different loop

2022-08-03 21:44:38.041 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.airthings_wave
2022-08-03 21:44:48.062 INFO (SyncWorker_0) [custom_components.airthings_wave.sensor] Found 1 airthings device(s)
2022-08-03 21:44:48.157 INFO (SyncWorker_0) [custom_components.airthings_wave.airthings] Not able to connect to 04:EE:03:BB:B4:13
Traceback (most recent call last):
  File "/config/custom_components/airthings_wave/airthings.py", line 239, in connect
    ret = await self._dev.connect()
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/client.py", line 107, in connect
    device = await BleakScannerBlueZDBus.find_device_by_address(
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/scanner.py", line 221, in find_device_by_address
    return await cls.find_device_by_filter(
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/scanner.py", line 250, in find_device_by_filter
    async with cls(detection_callback=apply_filter, **kwargs):
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/scanner.py", line 96, in __aenter__
    await self.start()
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/scanner.py", line 137, in start
    self._stop = await manager.active_scan(
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/manager.py", line 376, in active_scan
    reply = await self._bus.call(
  File "/usr/local/lib/python3.10/site-packages/dbus_next/aio/message_bus.py", line 305, in call
    await future
RuntimeError: Task <Task pending name='Task-3555' coro=<AirthingsWaveDetect.get_info() running at /config/custom_components/airthings_wave/airthings.py:259> cb=[_run_until_complete_cb() at /usr/local/lib/python3.10/asyncio/base_events.py:184]> got Future <Future pending> attached to a different loop
2022-08-03 21:44:48.170 ERROR (SyncWorker_0) [custom_components.airthings_wave.airthings] Not getting device info because failed to connect to device.

[lone-cloud]

I've been debugging on a fork (https://github.com/lone-cloud/sensor.airthings_wave) and I'm getting the exact same error. I would definitely agree that it's the root cause of the issue here. The eception implies that there are multiple threads running and python doesn't seem to like when variables set in one thread get re-assigned in another. I have not been able to figure it out yet.

[B1ob]

In my opinion we should adapt this component to the new bluetooth integration. However i'm quite busy at the moment and also lacking the experience here.

[MartyTremblay]

Tons of hints on how to update to bleak 0.15.0 here https://github.com/home-assistant/core/commit/80a9659524a47868070a16cdfa7f654fba7f6bb7

[mjmccans]

While I have not done much digging yet, I wonder if this is potentially related to the issue addressed by flecmart's PR. I wonder that because I have tried the dev branches of my own related scripts (here and here) against Bleak 0.15.1 and they work without issue, but I did implement flecmart's changes to ensure that disconnect is always called. It may be worth looking into that angle.

[lone-cloud]

@mjmccans I don't understand how anything works for you because using bleak 0.15.1 will result in the same bleak version mismatch that we started out with. edit: I noticed that a new HA patch just came out (for me) which is on bleak 0.15.1. This custom component really needs to migrate to using the new bluetooth integration instead of calling bleak directly, as was suggested, otherwise we'll keep running into this issue.

flecmart's PR didn't solve the issue, but I noticed that it broke exception handling because he was raising exceptions that didn't originate from the base Exception class. You'll see a very unhelpful "exceptions must derive from BaseException" errors in the log without fixing the new raises.

I did look over https://github.com/home-assistant/core/commit/80a9659524a47868070a16cdfa7f654fba7f6bb7 but I couldn't find anything relevant to the issue here. Saying that, I'm a (mostly) typescript dev and this is my first time playing with python and with HA custom integration dev in general. I'm now at my wit's end about this got Future <Future pending> attached to a different loop error and someone more familiar with python+asyncio should probably take a look at this point.

[mjmccans]

@lone-cloud I will start with a disclaimer that I am not expert in this area, so please keep that in mind when reviewing anything I say here. I did the initial conversion from Bluepy to Bleak, but I definitely made mistakes along the way (some of which may not have been found yet). In terms of my scripts, one is a script that is meant to run on a machine separate from Home Assistant and it would have its own version of Bleak. The use case for me was to have the script running on a Raspberry Pi closer to the Airthings device which would in turn send the sensor values to Home Assistant using MQTT. The other is that same script packaged as a Home Assistant add-on, which I made when I transitioned to Home Assistant OS. The add-on runs in its own container and also has its own version of Bleak, and Home Assistant exposes the DBUS socket to the container for communications with the bluetooth adapter. What I find is interesting is that these scripts appear to have no issue running with their own copies of Bleak 0.15.1. The reason I thought flecmart's PR could be in play was the remark about this looking like a potential threading issue, and that was also the only difference between my version of airthings.py and the one in this repository.

I wonder if the issue is that the new bluetooth integration is not playing nice with this custom integration and that is causing the issue. Ultimately @B1ob is likely correct that the right answer is to move to the new bluetooth integration, but it would be interesting to see if the issue goes away if you uninstall or disable the new Home Assistant bluetooth integration. That could help debug this issue, or show that I am completely out to lunch.

[lone-cloud]

I just re-tested on the latest HA with bleak 0.15.1 and the new bluetooth integration removed. It appears that the issue is indeed because of the new bluetooth integration. Unfortunately, when it's disabled this component still fails to connect due to Event loop is closed... new error message though, yay. I've been trying to create and assign a new event loop via asyncio, but to no avail. HA somehow has control of it and I haven't been able to figure out how to get it working...

[mjmccans]

Well, I guess that is progress. Probably obvious, but this link has some details and best practices for using the new bluetooth integration. It is not urgent for me to dig into at the moment because my scripts are still working, but I am interested in the topic and will try to dive in in the future. From the documentation it actually looks like it would be pretty easy to get auto-discovery working, which would be pretty useful.

[flecmart]

I was trying to find some documentation about how to use the new bluetooh integration as a developer. Do we even need a custom component after integrating the airthing devices with the new integration?

[B1ob]

Do we even need a custom component after integrating the airthing devices with the new integration?

Yes, as i understand it the integrations rely on the new bluetooth integration as a kind of middleware.

As stated on the bluetooth integration page under "Integrations that require exclusive use of the Bluetooth Adapter" this does not work in parallel with legacy integrations. I also thought it could be possible to introduce "async" to the sensor.py. However i still think adapting to the new integration is the best approach.

[vincegio]

I've played around a with this I think we need to wait for https://github.com/home-assistant/core/pull/76342 to be merged as it adds polling possibilities to the new system.

Anyways, here's some of my progress :tada:

first time working on home assistant integrations :baby:

[lymanepp]

API changes coming in 2022.9 - https://developers.home-assistant.io/blog/2022/08/08/bluetooth-api-changes/

[hubertron]

Like the others I have the same issue. Actually updated from Feb update.

[DavidS]

I downgraded back to 20.22.7 for now. Thank $deity for backups :-D

[blackest]

Back to 2022.7.7 for me also (HA 2022.8.3 is using bleak 0.15.1) HA 2022.8.3 breaks more than just this component and it probably doesnt help i'm using python 3.9x as well.

Funny coincidence bleak 0.15.1 development was sponsered by nabucasa.com

[BladeBear]

https://github.com/mjmccans/hassio-addon-airthings, this is working with Airthings wave with current Home Assistant release.

[MartyTremblay]

Thanks for the info @BladeBear,

However, the project link you provided is an add-on and not an HA component which means that it doesn't depend on the same bleak instance as the one built into HA.

None the less, this MQTT approach clearly works and could be a viable solution until we get our component working again.

[lone-cloud]

I've gotten hard stuck on the new way to connect to BLE devices: BleakClient(bluetooth.async_ble_device_from_address(hass, mac)) The new docs say that the BleakClient must now be passed in a BLEDevice and the bluetooth.async_ble_device_from_address(hass, mac) resolves as expected, but when passing it into the BleakClient it just hangs forever for me with no debugable trace. Not sure if anyone's run into this issue. I searched through the hassio core integrations, but nowhere else do they try to use the BleakClient, other than the bluetooth integration.

[Jc2k]

HA integrations in core have to push some of their logic for talking to a device to a library on pypi. So what you'll probably find is stuff like this where the bledevice is passed to an external library and that is where the BleakClient lives.

There will also be logic in there to handle if the BLEDevice changes (via bluetooth.async_register_callback maybe). For example, in a future release you can have remote proxies (remote bluetooth dongles over usbip or EPSHome etc). AIUI, the BLEDevice might change as it self optimises the best proxy to use for each device.

What does the output from async_ble_device_from_address look like for you?

I don't know if it will help but make sure that your custom component depends on the bluetooth integration in its manifest.json. What does your logging config look like? And what bluetooth adapter are you using (we are having a haaard time with realtek atm).

[vincegio]

I started working on a draft to try and get this into core. https://github.com/vincegio/airthings-ble

That's inspired from the Bluetooth devices repos, but since there are no advertisements, it's closer to PySwitchbot. Got some minor help from bdraco where to find the docs and how to think around this.

It works and the core component is kind of done. But haven't been able to commit it to my fork due to pre-commit errors for an unrelated file. Anyways, that's irrelevant for this issue though :)

[lymanepp]

I started working on a draft to try and get this into core. https://github.com/vincegio/airthings-ble

Very clean!

[vincegio]

Thanks! Here's the commit in core. Still got some stuff to do but it works https://github.com/vincegio/core/commit/a07d07b5f268c73e6d7962b1088fca6edcbba3a1

[B1ob]

Hi @vincegio nice work :) i had a first look at your integration. My feedback is the following:

• There is a bug in _get_service_characteristics which is missing a "return device" statement.
• I would add the address (mac) to the enity_id as with multiple devices the id would not be unique anymore (Or ask the user during config flow for a unique name).

[vincegio]

Hi @vincegio nice work :) i had a first look at your integration. My feedback is the following:

• There is a bug in _get_service_characteristics which is missing a "return device" statement.
• I would add the address (mac) to the enity_id as with multiple devices the id would not be unique anymore (Or ask the user during config flow for a unique name).

Thank you :+1: The entity part makes sense, thought it'd just add _2 at the end or something.

[lone-cloud]

Does anyone know if there's a simple way to use vincegio's component before it gets merged into core? I'm guessing someone would need to create a new repo, copy and paste his code in and add that repo in HACS? Is that correct? Also @vincegio will your component support the battery % for the airthings mini? I'm using one if there's something I could help to test.