core
core copied to clipboard
home-assistant
New Bluetooth integration
The problem
Discovers Bluetooth after update, enable it then get this error
Config entry 'Bluetooth' for bluetooth integration not ready yet: Failed to start Bluetooth: [org.bluez.Error.InProgress] Operation already in progress; Retrying in background
Try manually adding to config, same error, left overnight, same error
What version of Home Assistant Core has the issue?
2022.8.0
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant OS
Integration causing the issue
Bluetooth
Link to integration documentation on our website
No response
Diagnostics information
Config entry 'Bluetooth' for bluetooth integration not ready yet: Failed to start Bluetooth: [org.bluez.Error.InProgress] Operation already in progress; Retrying in background
Example YAML snippet
No response
Anything in the logs that might be useful for us?
Config entry 'Bluetooth' for bluetooth integration not ready yet: Failed to start Bluetooth: [org.bluez.Error.InProgress] Operation already in progress; Retrying in background
Additional information
No response
![probot-home-assistant[bot] avatar](https://avatars.githubusercontent.com/in/36093?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hey there @bdraco, mind taking a look at this issue as it has been labeled with an integration (bluetooth) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)
@tonyt2975 @EvgenaNet
I SSH'd in and used the command
bluetoothctl scan on
and restarted home assistant for it to work
@tonyt2975 @EvgenaNet I SSH'd in and used the command
bluetoothctl scan onand restarted home assistant for it to workafter rebooting HA the problem comes back
Hmm, I didn't reboot my machine after. I'll have to figure out how to enable scan on boot. I'll look into it again tomorrow night.
@tonyt2975 @EvgenaNet I SSH'd in and used the command
bluetoothctl scan onand restarted home assistant for it to workafter rebooting HA the problem comes back
Hmm, I didn't reboot my machine after. I'll have to figure out how to enable scan on boot. I'll look into it again tomorrow night.
after rebooting HA the problem comes back
Sometimes I have to use bluetoothctl to run power off, power on, scan on to unstick bluez on the host wen I get that error. However I think (haven't proven) that bluetoothctl stops the scan when it exits. I also have to do this when i restart HA.
I currently think I have a buggy version of bluez OR something else is fighting for control of bluetooth. I haven't had time to dig in yet though.
I am on debian bullseye. My bluez version is quite a few versions behind the latest upstream release, but there isn't an official route to upgrade it. Anyone using Home Assistant OS, what version are you running? And what bluetooth adaptor are you using? Are you on Arm or x86_64?
Sometimes I have to use bluetoothctl to run
power off,power on,scan onto unstick bluez on the host wen I get that error. However I think (haven't proven) that bluetoothctl stops the scan when it exits. I also have to do this when i restart HA.
Thanks! That worked perfectly, fixing the issue, running:
And then reloaded the Bluetooth integration. Immediately it found my Switchbot curtains, the Xiaomi kettle, and hygrometer. Super happy! 👍
I am using a VMware player (HAOS VM) and was having this error or similar when trying to setup bluetooth. With bluetoothctl scan on and the below edit of *.vmx config file to add the VID:PID I was able to get Bluetooth integration started.
usb.autoConnect.device0 = "0xYourVID:0xYourPID"
error (Retrying setup: Failed to start Bluetooth: [org.bluez.Error.InProgress] Operation already in progress)
It didn't survive a reboot via HA. The BT adapter is lost by the VM on reboot Cold boot with integration previosly setup ---> start bluetoothctl scan on via terminal addon and it will setup again.
Not that this is pertinent. It still won't connect to my Inverter via addon. I can scan and see my Inverter in my addon with Bluetoothctl. Yay I actually got it to connect!
VID PID links https://community.home-assistant.io/t/virtual-box-cpu-cores/432535/2 https://smarthomepursuits.com/how-to-automatically-connect-usb-drives-to-vmware-workstation-on-power-on/
Also in terminal addon (although I haven't setup ssh -rsa key yet) [core-ssh ~]$ rfkill list all rfkill: /dev/rfkill: Operation not permitted
Works perfect on my pi3 HAOS and pi4(supervisor) so far. Addon works on those too.
Useful for reseting dead Bluetooth. Although that was on debian 11 not haos
modprobe btusb
systemctl restart bluetooth
This was on Debian VM so Bluetooth was starting dead on reboot... Don't ask me how I found it I don't know.
It was related to VMX config. below was originally set to false. So bluetooth was being ignore on Boot.
usb.restrictions.defaultAllow = "TRUE"
profiles/sap/server.c:sap_server_register() Sap driver initialization failed.
This fixed it https://raspberrypi.stackexchange.com/questions/40839/sap-error-on-bluetooth-service-status
Had this error on Deb VM until Bluetoothctl scan on started error (Retrying setup: Failed to start Bluetooth: [org.bluez.Error.InProgress] Operation already in progress)
once scanning it setup.
Frustrating one indeed. In the end I purchased a USB adaptor and that works. Doesn’t reveal what is going on with the Pi 4 internal Bluetooth but at least allows me to use the integration. Thanks for all for the suggestions, hopefully some solutions worked for others in a similar situation.
Using Bluetooth Low Energy Monitor from HACS solve the problem...
Where i can get it?
same here. BLE from HACS worked fine, but the new bluetooth integration doesn't. the new integration will soon be way ahead the HACS BLE Integration. so why closing this issue? it's a bug that needs to be fixed.
I get the same error after inserting an old Microsoft Bluetooth adapter (Broadcom BCM2045A) that I had lying around on my server, passing it through the Home Assistant OS VM in ESXi 6.7, and restarting HA. The Bluetooth integration loads, but shows the same error.
I did use bluetoothctl scan on and restarted HA, which got rid of the error for the moment. I'm still waiting to see if any of my Bluetooth devices are discovered.
If you run bluetoothctl on its own it will drop you into a shell, where you can type scan on but then see live data from the device. It's a great way to help isolate the problem. No data when you do that, then the problem is not with HA.
I currently see:
[bluetooth]# power off
Changing power off succeeded
[CHG] Controller 44:03:8E:6A:E2:39 Powered: no
[CHG] Controller 44:03:8E:6A:2:39 Discovering: no
[bluetooth]# power on
Changing power on succeeded
[CHG] Controller 44:03:8E:6A:E2:39 Powered: yes
[bluetooth]# scan on
Discovery started
[CHG] Controller 44:03:8E:6A:E2:39 Discovering: yes
If I run scan on at the following prompt, I see the same error text that I saw in the HA Devices & Services screen.
I navigate back to Devices & Services in HA, and the integration does not show an error at that point.
If you run
bluetoothctlon its own it will drop you into a shell, where you can typescan onbut then see live data from the device. It's a great way to help isolate the problem. No data when you do that, then the problem is not with HA.
i'm using rpi4 without usb adapter... shouldn't that thing work out of the box ?
So on a working bluetooth stack it will stream stuff like this:
[CHG] Device C4:7C:8D:6A:3E:7A ServiceData Key: 0000fe95-0000-1000-8000-00805f9b34fb
[CHG] Device C4:7C:8D:6A:3E:7A ServiceData Value:
71 20 98 00 c0 7a 3e 6a 8d 7c c4 0d 04 10 02 f2 q ...z>j.|......
00 .
[CHG] Device 6A:BC:E1:50:62:41 RSSI: -60
[CHG] Device 6B:B4:78:20:33:D6 RSSI: -74
[CHG] Device 6B:B4:78:20:33:D6 RSSI: -62
If you don't see that in the OS bluetoothctl, then HA doesn't see that, and nothing will work.
I run my prod environment using the official HA container but on a Debian Bullseye host with an external USB bluetooth adapter thats known to work, only it doesn't for me. I'm in the process of backporting BlueZ packages to try and find out if thats the problem. Thats after trying different usb ports because e.g. usb3 is known to cause problems for bluetooth etc.
As for rpi4 - we are actively encouraging people not to use onboard bluetooth because the range is known to be crap. As for reliability, I have no idea. I haven't been able to get hold of one to use for HA testing.
@Jc2k "we are actively encouraging people not to use onboard bluetooth because the range is known to be crap."
Could you give some reference please?
Not really, its just experience. Seeing lots of users pre-emptively going with external because they known they have sub optimal RF conditions in their hosting environment. Noticing others having errors which are obviously range related, advising them to try an external dongle and it helped, etc.
An external dongle is not a cure all of course, especially some of the "nano" ones, and if i put one in the back of my PC its still like a faraday cage under my desk, and i can't see my most distance MiFlora. With an extension to lift it 2m off the ground and it can.
But let me rephrase it a bit. Any internal bluetooth is in general at much higher risk of leading to a crappy user experience, as it is much more likely to be in conditions that are known to degrade 2.4ghz radio signals. More likely to be in a case that acts as an RF shield, more likely to be too close to noise generating USB devices, more likely to be suboptimally located (in a rack, under your desk, etc).
"Crap" probably is unkind, and was more to stress the point that nothing in the worrld is guaranteed to work out of the box. If you optimally locate a completely naked pi4, don't use any usb devices, only use 5ghz wifi or ethernet, etc the range is probably fine.
Some of this is touched in the documentation about how to reduce the impact of interference,
https://www.home-assistant.io/integrations/bluetooth/#bluetooth-interference-with-other-devices
Not sure if its related, I get the same error in the log,? I am running HA in a VM with a USB device passed through, every time i backup (nightly) the hypervisor disconnects the usb device, I do not think HA reinitiates the attached usb dongle again? is there are way to automate the power on and scan?
Home Assistant 2022.8.2 Supervisor 2022.07.0 Operating System 8.4 Frontend 20220802.0 - latest
Logger: homeassistant.components.bluetooth Source: components/bluetooth/init.py:422 Integration: Bluetooth (documentation, issues) First occurred: 8:38:34 am (1 occurrences) Last logged: 8:38:34 am
Error stopping scanner: [org.bluez.Error.Failed] No discovery started
Logger: homeassistant.config_entries Source: components/bluetooth/models.py:89 First occurred: 9:07:28 am (1 occurrences) Last logged: 9:07:28 am
Error unloading entry Bluetooth for bluetooth Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 496, in async_unload result = await component.async_unload_entry(hass, self) File "/usr/src/homeassistant/homeassistant/components/bluetooth/init.py", line 282, in async_unload_entry await manager.async_stop() File "/usr/src/homeassistant/homeassistant/components/bluetooth/init.py", line 583, in async_stop self._cancel_device_detected() File "/usr/src/homeassistant/homeassistant/components/bluetooth/models.py", line 89, in _remove_callback self._callbacks.remove(callback_entry) ValueError: list.remove(x): x not in list
** update **
Yes I can confirm this is the case
Backup started, usb disconnected
Backup complete
Scanning not automatically reinitiated, but i can issue the 'scan on' and its start again, i just need to automate this, but i do not know how :-(
On bluetoothctl commands I get
for power on or power off org.bluez.Error.Busy
for scan on org.bluez.Error.NotReady
Also, if I run sudo rfkill unblock all (or any rfkill command) I get Operation Not Permitted
Any suggestions what to do?
I've encountered the same thing while using an ASUS BT400 (on a NUC-like system) and the internal Pi 3B Bluetooth. On the Pi 3B, if I try to bring things back to life using bluetoothctl, I get:
Error stopping scanner: [org.bluez.Error.InProgress] Operation already in progress
# bluetoothctl power off
Changing power off succeeded
# bluetoothctl power on
Failed to set power on: org.bluez.Error.Failed
#
I didn't have a problem with the BTLE integration.
Hi, I've currently using Home Assistant 2022.8.3 but I'm having a lot of issues with the Bluetooth devices. Govee devices seem to be working well... but Xiaomi BLE refresh "every once in a year"... when using Passive BLE monitor integration from HACS, everything worked flawless.
In addition, I could not pair Xiaomi with the pvx firmware so I had to "rollback" to original firmware. Is it me? Am I doing somehting wrong? Is there a way to fix this? Or should I keep using BLE from HACS while everything gets solved?
The Xiaomi integration is for firmware provided by Xiaomi or firmware written to broadcast exactly like Xiaomi firmware. If PVX firmware is the same thing as ATC firmware, thats known to not be supported. Every parser in ble monitor will eventually be converted to a full integration in HA but that requires people to do the work. I did a MiBeacon one (i.e. xiaomi_ble) because I have a bunch of Mi Flora plant sensors.
The stock firmware for LYWSD03MMC has a very slow broadcast rate. It might be as slow as once every 20 minutes? Certainly no faster than once every 10 minutes. Unfortunately i don't have one, so it only works at all thanks to one very helpful beta tester. One of the things the ATC firmware does is massively increase the transmission rate. Not only does that mean quicker updates, but if some packets are lost because you are barely in range, you don't notice because there are plenty more. I.e. if there is only 1 broadcast every 10 mins and you miss it... Your graph is a mess.
I.e. this guy is right next to my bluetooth:
But this plant is really out of range:
Like i said, this was probably masked for you by ATC ramping up the broadcast rate. Right now you are probably better sticking with BLEM until someone makes an ATC integration.
I have the same problem @escobarin3
Home Assistant 2022.8 (.0 .1 .2 .3) with the new BLE integration, doesn't update my two Lywsd03mmc as often as when using HACS BLE add-on.
With HACS BLE, the update append every 10 minutes (if there is a change), but with the new HA 2022.8 BLE integration the update can take 1 or 2 hours ... sometimes 20 minutes ... completly random.
// stock firmware, RPI 4.
On
bluetoothctlcommands I get forpower onorpower offorg.bluez.Error.Busy forscan onorg.bluez.Error.NotReadyAlso, if I run
sudo rfkill unblock all(or any rfkill command) I get Operation Not PermittedAny suggestions what to do?
Having the same issue here on stock hardware Rpi4 internal Bluetooth. BT-Integration is stuck with error org.bluez.error.notready, ssh commands the same as above. Already tried to deactivate all dependent itegrations (HomeKit Controller and Switchbot in my case but no results)
Any suggestions?
2022.8.5 and still the same error:
Error stopping scanner: [org.bluez.Error.InProgress] Operation already in progress
The integration is incomplete and glitchy.
That error probably means Bluetooth on your host has got stuck and is rejecting requests from HA. You will probably get the same errors if you use bluetoothctl to directly interact with your dongle. HA can't do anything about it. It will be retrying in the background in the hopes that someone fixes bluez. Unfortunately sometimes the dongles crash so bad you need to unplug them to reset them.
That error probably means Bluetooth on your host has got stuck and is rejecting requests from HA. You will probably get the same errors if you use bluetoothctl to directly interact with your dongle. HA can't do anything about it. It will be retrying in the background in the hopes that someone fixes bluez. Unfortunately sometimes the dongles crash so bad you need to unplug them to reset them.
The fact is that the same dongle works flawlessly with BLE Monitor. Apparently, it is still a matter of HA integration.
That doesn't use bluez. Feel free to continue using BLEM if you don't have a dongle that works well with bluez.
That doesn't use bluez. Feel free to continue using BLEM if you don't have a dongle that works well with bluez.
I wonder why the HA integration then uses such a glitchy bluez that does not work correctly with bluetooth adapters of popular mini pc?
BlueZ is the Official Linux Bluetooth protocol stack http://www.bluez.org/
There really aren't any other realistic choices for complete bluetooth implementations on linux.
@Jc2k, hello.
That error probably means Bluetooth on your host has got stuck and is rejecting requests from HA. You will probably get the same errors if you use bluetoothctl to directly interact with your dongle. HA can't do anything about it. It will be retrying in the background in the hopes that someone fixes bluez. Unfortunately sometimes the dongles crash so bad you need to unplug them to reset them.
My (custom) integration is using BlueZ (bleak) to communicate with device. It work fine if bluetooth integration is disabled. And got Operation already in progress from startup while connecting to device if the bluetooth integration is enabled.
systemctl restart bluetooth is fixing it for a short time, but then Operation already in progress while connect again.
It's probably the same as https://github.com/home-assistant/core/issues/76839. Does your integration run a BleakScanner instance 24/7, or does it just connect to a known mac address?
Yes, it looks similar.
I'm not using BleakScanner. Just direct connections:
- connect -> send command -> read response from notification -> disconnect every minute to track current state;
- connect -> send update command -> disconnect to change device state.
Right. I can think of 2 seperate things going on here.
So the current theory behind most of the problems in this thread is that there are sometimes "broadcast storms" (lots of BLE devices happens to send an advertisement at about the same time) and this is overwhelming a buffer in a kernel driver or bluez or something. Probably the kernel because this is most common in realtek devices. But this only happens when we are listening for advertisements (which we need to do for passive sensors). So if you never used BleakScanner your integration probably never triggered this bug in your bluetooth dongle/kernel before.
A newer kernel might fix it (we've had some reports of this but not enough to be confident). The best HA can do to fix this is some hacky auto-restarter job.
The other possibility in your case is that when you connect to a MAC address, bleak actually secretly starts a BleakScanner in the background to resolve the MAC address. You can actually only run a single scanner at once with BlueZ, so if the bluetooth integration is already running one, your integration can't resolve the MAC address. That could explain why your integration gets an "Operation already in progress" error.
It might be enough for your custom integration to add a bluetotoh to its "dependencies" key in its manifest.json. This will make sure that the bluetooth integration has chance to set up its compatibility shims before your code runs. Failing that, you want to use the async_ble_device_from_address API (https://developers.home-assistant.io/docs/network_discovery?_highlight=bluetooth#fetching-the-bleak-bledevice-from-the-address). This will resolve your MAC address into a BLEDevice without starting an extra bleak scanner. Anywhere you pass a MAC address to Bleak, pass a BLEDevice instead. Bonus, this will be faster as HA will have pre-resolved the MAC address.
If you try this and have trouble with it please start another ticket to track it, as it is a seperate problem to the first one i describe.
Thank you for explanation. Work fine with async_ble_device_from_address.
I am also seeing this same "bug" i have to unplug Bluetooth adapter and plug back in to solve issue. but every couple days it stops working. I am using KNOWN WORKING adapter ZEXMTE BT-505 (Long Range) and tried XDO BT802 (Long Range) both have same issue.
I am also seeing this same "bug" i have to unplug Bluetooth adapter and plug back in to solve issue. but every couple days it stops working. I am using KNOWN WORKING adapter ZEXMTE BT-505 (Long Range) and tried XDO BT802 (Long Range) both have same issue.
That sure sounds like the realtek firmware problem discussed here https://github.com/home-assistant/core/issues/76850#issuecomment-1217708223
That's the same one I'm using for my production system but I'm running haos dev (9.0.dev20220816) though which has a newer kernel so that might be why I'm not having any issues anymore (or I just haven't gotten unlucky yet)