core icon indicating copy to clipboard operation
core copied to clipboard

Bluetooth integration causes high CPU usage

Open chrisx8 opened this issue 2 years ago • 9 comments

The problem

System CPU usage is consistently high when the bluetooth integration is enabled. I tested with Bluetooth enabled and disabled.

I'm running Home Assistant OS on Proxmox, with an Intel 9260 Bluetooth adapter passed through as a USB device. As reported by Proxmox, the CPU usage (of the VM) shows 12-15% hourly average with bluetooth enabled, and 3-5% with bluetooth disabled.

What version of Home Assistant Core has the issue?

2022.11.1

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Bluetooth

Link to integration documentation on our website

https://www.home-assistant.io/integrations/bluetooth/

Diagnostics information

No response

Example YAML snippet

None. Enabling the Bluetooth integration causes this issue to occur.

Anything in the logs that might be useful for us?

No relevant log messages

Additional information

Diagnostics of bluetooth suggests it has discovered over 30 different devices.

chrisx8 avatar Nov 06 '22 02:11 chrisx8

Hey there @bdraco, mind taking a look at this issue as it has been labeled with an integration (bluetooth) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of bluetooth can trigger bot actions by commenting:

  • @home-assistant close Closes the issue.
  • @home-assistant rename Awesome new title Change the title of the issue.
  • @home-assistant reopen Reopen the issue.
  • @home-assistant unassign bluetooth Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)


bluetooth documentation bluetooth source (message by IssueLinks)

home-assistant[bot] avatar Nov 06 '22 02:11 home-assistant[bot]

Please post a callgrind file from the profiler.start service

bdraco avatar Nov 06 '22 03:11 bdraco

Done! Enabled Bluetooth, restarted HA, and ran profiler.start with seconds: 10 once startup completed.

callgrind.out.1667704104081537.txt

chrisx8 avatar Nov 06 '22 03:11 chrisx8

Can you do one with the default 60 seconds ?

bdraco avatar Nov 06 '22 03:11 bdraco

callgrind.out.1667704677477982.txt

Re-ran with 60 seconds

chrisx8 avatar Nov 06 '22 03:11 chrisx8

It looks like your core instance is ~96.75% idle and is processing ~7000 bluetooth advertisements per minute. Thats a bit high but it looks like the cpu drain isn't in the core process.

callgraph

Which version of HAOS are you running?

Can you check how much CPU time bluetoothd is using?

bdraco avatar Nov 06 '22 03:11 bdraco

I'm running HA OS 9.3, on Proxmox 7.2 (latest).

Not sure how I could check CPU time, but I'm attaching 2 screenshots of top running on HAOS, one with Bluetooth enabled and one with Bluetooth disabled.

Bluetooth enabled: top_with_bluetooth

Bluetooth disabled: top_without_bluetooth.png

chrisx8 avatar Nov 06 '22 04:11 chrisx8

It looks like its the dbus overhead that's causing the higher cpu as bluetoothd, pulseaudio, and home assistant are all heavy dbus users talking to dbus-broker. At 7000 advertisements per minute, thats going to be a lot to process for any system (even with my desk littered with test bluetooth devices, my Odroid n2+ is only processing ~4500-5000/min).

dbus-broker is the fastest dbus option out there so it there likely isn't anything we can do to improve the performance.

It is a bit surprising though that an RPI4b can process more advertisements and end up with a lower cpu % calculation so there might be something else going on with the system

Its also possible you might have a single device that is flooding your system with advertisements. You might take a look at running bluetoothctl and see if you can pin point a specific device.

bdraco avatar Nov 06 '22 04:11 bdraco

Interesting that an RPi4 can do better... I'll look into this a bit more and test on some different systems, but I'm suspecting this might have something to do with Proxmox or my Bluetooth adapter.

Thanks for looking into this!

chrisx8 avatar Nov 06 '22 04:11 chrisx8

I installed Debian 11 directly on the same host and tested this again with both HA Core and HA Container. I'm seeing very similar CPU usage patterns as the OS running under Proxmox.

I also ran bluetoothctl on the host system and tried to find any misbehaving devices. As soon as I launch bluetoothctl, I get a ton of messages like

[CHG] Device XX:XX:XX:XX:XX:XX RSSI: -71

There doesn't seem to be any particular patters, as they all seem to point to different devices (different MAC address).

Question: Are there any differences in CPU usage with different Bluetooth adapters like Intel vs. Broadcom (on the Pi 4) vs. Realtek (on the Odroid N2+)?

chrisx8 avatar Nov 10 '22 15:11 chrisx8

If you are running Debian 11 you probably have the slow dbus broker, unless you manually installed the one HAOS uses.

Jc2k avatar Nov 10 '22 17:11 Jc2k

I have switched to dbus-broker following the instructions

chrisx8 avatar Nov 10 '22 17:11 chrisx8

We did manage to squeeze another 10-20% out of the dbus code in 2022.12.x but the rest is python overhead that we likely cannot optimize away

bdraco avatar Nov 12 '22 17:11 bdraco

There have been some additional optimizations in newer versions but there isn't much more we can do here as all the big optimization wins in the code paths have already been realized so I'm going to close this issue.

One quick note though, if you have HA OS 9.4+ you'll get the option to switch to passive scanning in the bluetooth integration options flow which can significantly reduce the number of packets that need to be processed.

bdraco avatar Jan 07 '23 21:01 bdraco

We did manage to get some more performance out of it in 2023.8.x. There are probably still some more wins in the pipeline if we can get https://github.com/Bluetooth-Devices/dbus-fast/pull/213 working

Also if you switch to using ESPHome proxies, it should reduce the load on the local system significantly https://esphome.github.io/bluetooth-proxies/

bdraco avatar Jul 27 '23 23:07 bdraco