[Bug]: Discovered sensors do not restore state after a restart

[Ernst79]

What happened?

When sensors are discovered, they do not restore their state after a restart of HA, even if this is the default BLE monitor setting. This is caused by the fact that the HA config entity is not aware of the newly added sensors yet.

Opening the BLE monitor settings and clicking submit will add the MAC's to the config option devices and will immediately restore the state, after which it does work. However, this requires that if an sensor is added, you will have to open the BLE monitor options to make the restore state function to work.

Sensor type

All

Relevant log output

No response

[andrewjswan]

How about yaml mode? After restart all my sensor have unavailable state, after some time, have correct states ...

[Ernst79]

If you don't have the MAC (or UUID) configured in yaml, restore state won't work at the moment and you will see "unavailable" after a restart. However, if you add the MAC (or UUID) to yaml, restore state works normal.

So, if you have it configured like this, it won't work

ble_monitor:
  discovery: True
  restore_state: True

However, if you add the MAC (or UUID) it will work for the two specified sensors.

ble_monitor:
  discovery: True
  restore_state: True
  devices:
    - uuid: 'A495BB10-C5B1-4B44-B512-1370F02D74DE'
    - mac: 'A4:C1:38:74:B0:85'

I will try to fix this in a future release, but I'm a bit busy at the moment. In the meantime, you can use this workaround.

[Ernst79]

I just noticed that this has always been the case, the docs mention the following about restore_state

This option will, when set to True, restore the state of the sensors. If your devices are configured with a mac address, they will restore immediately after a restart of Home Assistant to the state right before the restart. If you didn't configure your devices, the state will be restored upon the first BLE advertisement being received.

Anyway, this issue is to improve the behavior, to also restore the sensors that are not configured with their MAC address. In UI mode, this means that you will have to open the BLE monitor options once after a sensor is discovered

[andrewjswan]

@Ernst79 My Yaml config:

# Mi BLE Devices
decimals: 1
period: 60
log_spikes: False
use_median: False
active_scan: False
restore_state: True
hci_interface: 0
devices: !secret ble_devices
report_unknown: False

!secret ble_devices look like:

ble_devices:
  - mac: 'XX:XX:XX:XX:XX:XX'
    name: 'xxxxxxxxxxxxxxxxx'
    encryption_key: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
  - mac: 'XX:XX:XX:XX:XX:XX'
    name: 'xxxxxxxxxxxxxxxxx'
    encryption_key: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
  - mac: 'XX:XX:XX:XX:XX:XX'
    name: 'xxxxxxxxxxxxxxxxx'
    encryption_key: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
  - mac: 'XX:XX:XX:XX:XX:XX'
    name: 'xxxxxxxxxxxxxxxxx'
    encryption_key: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
...

[gromitn]

Same issue. I have 4 of these, set up like:

ble_monitor: discovery: True bt_auto_restart: True devices: - mac: 'XX:XX:XX:XX:XX:XX' name: 'Lounge' encryption_key: 'XXXXXXXXXXXXXXXXXXXX' temperature_unit: C decimals: 2 use_median: False restore_state: true

All come back disconnected after a restart, and given these things are so slow to update it's a mild irk.

[andrewjswan]

I have all the sensors are prescribed as I wrote above, but some restore the state and some do not, before all restored instantly on startup.

[Ernst79]

Strange, is there any difference in the format in YAML, e.g. did you use uppercase macs and lower case macs, which could explain why some do update, and some others not. Or per haps lower/upper case names?

[andrewjswan]

All in one format:

  - mac: 'XX:XX:XX:XX:XX:XX'
    name: 'Sensor Name'
    encryption_key: 'xxxxxxxxxxxxxxxxxxxxxxxxxxx'

mac - Upperacse, encryption_key - lower case, name like above ...

[Ernst79]

Hmmm, really strange. I'll try to replicate it, but I'm a bit busy right now (new house). So could take some time before I can look into it.

[andrewjswan]

Thanks, no problems :)

[chertvl]

Same. No matter I use YAML config or GUI, all entities goes to unavailable after restart and save that state until HA catch first ble-packet for device. For this reason it became impossible to use ESP BLE Gateway with auto-generated macs-list from template sensor, this sensor not contains any macs from unavailable sensors. I need to pick up any device and bring it close to HA server (internal bluetooth adapter). Last stable working version for me is 8.1.1, stayed on it.

[myhomeiot]

Last stable working version for me is 8.1.1, stayed on it.

This doesn't helps with restore states, but helps with auto-generated macs-list, this template sensor use device attribute identifiers for getting Passive BLE Monitor devices macs (works with latest versions of BLE Monitor like 8.6.3).

template:
  - binary_sensor:
      - name: BLE Gateway
        icon: mdi:bluetooth
        state: "{{ is_state('input_boolean.settings_ble_gateway', 'on') }}"
        availability: "{{ is_state('binary_sensor.ha_started', 'on') }}"
        attributes:
          devices: >-
            {% set devices = namespace(items = []) %}
            {% for s in states | selectattr('entity_id', 'search', '^(device_tracker|sensor).ble_') | map(attribute='entity_id') %}
              {% set devices.items = devices.items + [device_id(s)] %}
            {% endfor %}
            {% set ns = namespace(items = []) %}
            {% for s in devices.items | unique %}
              {% set ns.items = ns.items + [(device_attr(s, 'identifiers') | first)[1]] %}
            {% endfor %}
            {{ ns.items | unique | sort | join('') | replace(':', '') ~ (states('input_text.settings_ble_gateway_add_device') | replace(':', '') | trim) if is_state('binary_sensor.ble_gateway', 'on') }}

[chertvl]

this template sensor use device attribute identifiers

Great idea, thank you!

[andrewjswan]

@Ernst79 Hi! Any news? Or are you still busy?

[Ernst79]

Thanks for the reminder. I’m starting to get more time, so I will look into it.

[chertvl]

Hey, any news please?

[lerra]

I have the same issue :(

[Ernst79]

Hi all, sorry I kept you waiting so long. Have been busy with the new HA BLE integrations, I have tried to reproduce the issue twice, on two different systems? But I can't reproduce it at all. I restores the state nicely on my system.

So, we need to figure out what is the difference between my system and yours.

Can you all please provide the following information. I will give mine as example.

system: HA core in venv
configuration of BLE monitor: via UI
How did you set "restore state on restart": BLE monitor is set to use restore state on global level, devices are set to "use default"
Devices that work correct: xiaomi LYWSD03MMC with default firmware
Devices that don't work correctly: None

[50494554524F]

the thermometer seems working because ble packages sent from it are tons... try removing the battery, no restore state after rebooting, it is restored when the battery is back in

[50494554524F]

my configuration:

Home Assistant 2022.8.7
Supervisor 2022.08.5
Operating System 8.5
Frontend 20220802.0 - latest

ble_monitor 9.1.1

ble_monitor:
  discovery: false
  decimals: 2
  restore_state: true
  devices:
    - name: 'Xiaomi - Thermometer - Test'
      mac: 'MAC'
    - name: 'Xiaomi - Window Sensor - Test'
      mac: 'MAC'
      encryption_key: 'KEY'

the thermometer is using pvvx, atc format not encrypted...

[Ernst79]

the thermometer seems working because ble packages sent from it are tons... try removing the battery, no restore state after rebooting, it is restored when the battery is back in

The LYWSD03MMC is a temperature sensor with a 10 minute interval for sending data. It’s actually one of the slowest sensors (with stock xiaomi firmware). But I will try the battery test, just to be sure.

[andrewjswan]

Home Assistant 2022.8.7
Supervisor 2022.08.5
Operating System 8.5
Frontend 20220802.0 - latest
ble_monitor 10.0.0

BLE Config in Yaml:

ble_monitor:
  decimals: 1
  period: 60
  log_spikes: False
  use_median: False
  active_scan: False
  restore_state: True
  hci_interface: 0
  devices: !secret ble_devices
  report_unknown: False

Devices that work correct: None Devices that don't work correctly: All, LYWSD03MMC (Stock firmaware), MiScale, MiFlora, SJWS01LM, MCCGQ02HL, etc ...

[chertvl]

Home Assistant 2022.8.7
Supervisor 2022.08.5
Operating System 8.5
Frontend 20220802.0 - latest
ble_monitor 9.1.1

BLE Config mode - YAML:

discovery: false
restore_state: true
devices: ...

Devices that work correct: No one Devices that don't work correctly: LYWSD03MMC (ATC Firmware), MiScale, MiFlora, Xiaomi Toothbrush T500

[Ernst79]

I was able to reproduce the issue.

I noticed that you were all using YAML, which I wasn't.

I think it is related to the fact that you rename your device in YAML. @chertvl and @andrewjswan do you both use the name parameter to rename you sensors, like @50494554524F ?

  devices:
    - name: 'Xiaomi - Thermometer - Test'
    ...

[Ernst79]

For those who are interested. Somewhere in the code, we do a check that the prefix of the unique_id is equal to known unique id's.

https://github.com/custom-components/ble_monitor/blob/e9d9c9eb51a2da7c384975d1716540d77bffaf0c/custom_components/ble_monitor/sensor.py#L205

E.g. t_ for temperature. In UI mode, the unique_id is always t_MACADDRESS. In YAML mode, it also has the name in the unique_id, when you overwrite it with your own name. To get the prefix, I did removesuffix(KEY) where KEY = the MAC address.

https://github.com/custom-components/ble_monitor/blob/e9d9c9eb51a2da7c384975d1716540d77bffaf0c/custom_components/ble_monitor/sensor.py#L203

This doesn't work if you use YAML with your own device name. So, I added a removesuffix(dev.name), which should do the trick.

[Ernst79]

The issue should be fixed in 10.2.0. In someone can let me know if it worked, that would be great.

[chertvl]

@chertvl and @andrewjswan do you both use the name parameter to rename you sensors, like @50494554524F ?

Yep.

The issue should be fixed in 10.2.0. In someone can let me know if it worked, that would be great.

I did update from 8.1.1 (last version for me wich restore states) to 10.2.0 just now. First start on 10.2.0 - all states restored. Restarted HA again, second start - again good. No any 'unavailable' entities after restart now. I think we did it! Lets wait for @andrewjswan and @50494554524F results. Thank you, @Ernst79 !

[andrewjswan]

@Ernst79 Look like fixed. Thanks!

[Ernst79]

Great, sorry it took so long, but every time I tried to reproduce it, I wasn't able to reproduce the issue.

[50494554524F]

seems all ok to me too.... thanks