core
core copied to clipboard
home-assistant
Automations referencing non-existing devices do not work after installing 2023.11.0b1
The problem
After upgrading to 2023.11.0b1 from b0, multiple device IDs changed. This caused multiple automations to break. Devices that had their IDs changed were from both Zigbee2MQTT and ZwaveJS-UI. There may be devices from other integrations as well, but those are the only 2 that impacted me via automations.
What version of Home Assistant Core has the issue?
2023.11.0b1
What was the last working version of Home Assistant Core?
2023.11.0b0
What type of installation are you running?
Home Assistant OS
Integration causing the issue
Zigbee2MQTT, ZwaveJS-UI
Link to integration documentation on our website
No response
Diagnostics information
No response
Example YAML snippet
No response
Anything in the logs that might be useful for us?
2023-10-27 18:02:34.118 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Worker - Bedroom Low Light' failed to setup actions and has been disabled: Unknown device 'f559286722b67dfb86349bb409f3806e'
2023-10-27 18:02:34.134 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Presence - Arrived Home' failed to setup actions and has been disabled: Unknown device '491487c43244c1c27f2e48a961bad702'
2023-10-27 18:02:34.135 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Mode Trigger - Night' failed to setup actions and has been disabled: Unknown device '491487c43244c1c27f2e48a961bad702'
2023-10-27 18:02:34.182 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Worker - Master Closet Light v2' failed to setup actions and has been disabled: Unknown device 'f61ed5009889ffbb96884c8d9f4a60f9'
2023-10-27 18:02:34.827 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Lock - Do stuff when code used' failed to setup actions and has been disabled: Unknown device '02510f7e01be61614d6e50dcf497ce4c'
2023-10-27 18:02:35.737 ERROR (MainThread) [homeassistant.components.automation] Automation with alias 'Lighting - Office Button' failed to setup actions and has been disabled: Unknown device '3271e90d5dc11f6f439840dd4ac8af39'
Additional information
No response
with #102766 some further validations for automations has been added. you now see this validation results. could you please verify if the device id's are really not present?
Yes the device IDs that were present in the automation were no longer the correct ones - after looking up the new device IDs and copy/pasting the ID into the failing automation, the automation was able to save successfully.
I did some more digging. I pulled my oldest backup (unfortunately I only keep them for 7 days) and it looks like the "new" device IDs were already present back in core 2023.10.3 based on what I'm seeing in the device registry.
So the change of device IDs happened at some point in the past, but the automations are only now failing due to the PR @mib1185 listed.
So ultimately, at least for my case, this doesn't look like it was caused by this beta - but this beta is bringing the issue to the surface. Which is a good thing, but is probably worth calling out as there will likely be more people that discover they have broken automations after this version.
found this via google after updated to 11.0b2 (from 10.5). Agree with iridris to add the callout just so others are aware, but my biggest issue is not so much updating automations, but knowing what has changed and where all it exists. For example, an automation that uses the device condition or trigger will fail the validation and let me know, so that's easy enough to find and replace. But automations which are triggered by zwave events also use those device ids, but are not called out. So I'm not entirely sure what my best options are for updating that stuff short of going through my house and pressing all of my zwave switches to trigger various automations to see if they still work.
Do these missing device ids show up in a log somewhere from the zwave event trigger? Or any other suggestions for the best way to make sure I have everything updated and can sleep easy knowing things will work the next time I try to use them?
But automations which are triggered by zwave events also use those device ids, but are not called out.
Please give an example of such a trigger and how you did configure it
I changed the PR title to better reflect the problem, since it seems devices were not changed in 2023.11, it's previous changes that went unnotified that now cause errors
same here https://github.com/home-assistant/core/issues/103237 with helper devices.
I had two automations that were created at the same time, each with two devices. One of the two devices differed (button/switch) across the two automations. The other device (light) was the same.
Post-update, it was the light that "disappeared" from the automations; the buttons/switches were still referenced correctly. It seems very odd that the IDs would change in one automation but not the other.
Same issue with my Zigbee devices. I have to modify my automations.yaml and replace all the human readable deviceIds by bluddy unreadable uid. A real nightmare, this makes also the script totally unreadable for later support.
Even complex is to retrieve which uid was hidden by the human readable device ID :'(
Dom
I also don’t understand why they did this, if I write automation by hand, why are unreadable uids not tedious and where can I get them? Okay, you would have made them for those automations that are created through the ui, but you also ruined everything for me with .yaml! Thank you, this is user friendly, keep it up! (ง'̀-'́)ง
Two questions?
- Is there a way to avoid this ID change when update to 2023.11?
- When update and the ID changes, you can fix automations manually (task for "advanced" users) but... Will the IDs change every update or just this time?
Same problem as many others with automations becoming unavailable. Has anybody tried just deleting the broken automations and the recreating them?
Same problem as many others with automations becoming unavailable. Has anybody tried just deleting the broken automations and the recreating them?
We lost the link between "Human readable IDs" (entity_id and device_id). Yo don't have a full list of all the unreachable devices, neither a full description of all the automation scripts you made and optimized.
By example,I am not able to create a group of lights and use that group to turn_on and/or turn_off the "lights_group".
Thanks to God, I was just recreating my homeautomation, I don't how I would have reacted if everything was under HA control.
One thing is certain, before the next update, I will do a FULL BACKUP, and not only the backup proposed by the update UI.
Where, by whom are all the "non-regression tests" done ? If they tests had been passed, this version would never have corrupted so many installations.
Apprently downgrading to 2023.10 is possible and doesn't have unwanted side effects. How to identify whatever will be broken by upgrading to 2023.11 before upgrading to 2023.11 and what exactly is the problem and how to fix it?
If I understand the problem correctly, replacing all device IDs with the corresponding entity IDs will fix it. Can someone please confirm?
I have downgraded to 2023.10 and everything is fine. Now I'm looking forward to a fix in order to upgrade Any news about this issue on 2023.11.1? I don't see they talk about a solution in this new release.
Seems to be showing up for me also with helper entities and the select service
For automations this looks like a feasible solution for those prefering yaml over the GUI: https://community.home-assistant.io/t/2023-11-to-do-add-release-title/634647/239:
Don’t use device triggers. Use State triggers.
I have downgraded to 2023.10 and everything is fine. Now I'm looking forward to a fix in order to upgrade Any news about this issue on 2023.11.1? I don't see they talk about a solution in this new release.
I updated this morning so I assume I went from 2023.10.3 to 2023.11.1 since it was released yesterday. I experienced the same issue with automations not working: fortunately I always make a full backup and was able to go back to 2023.10.3 and the issue has gone away. I would say this isn't fixed in 2023.11.1 then.
@Campagne8758 please could you verify, if the as "unknown device" alerted devices are really back (with this exact id) after the rollback?
@mib1185 Ah sorry, I did not make a note of the specific device ids that appeared in the logs so I don't have the necessary information to double check, are you aware of a way to access the logs generated before I rolled back?
@Campagne8758 please could you verify, if the as "unknown device" alerted devices are really back (with this exact id) after the rollback?
For me, the device IDs remained unchanged which makes sense, as 2023.11 does not change device IDs, just adds some extra checks for device IDs.
are you aware of a way to access the logs generated before I rolled back?
it depends on how you did the rollback. HA copies always the last log file to homeassistant.log.1
if you have a full backup from 2023.11.1 than you could extract both log files it from there
For me, the device IDs remained unchanged which makes sense, as 2023.11 does not change device IDs, just adds some extra checks for device IDs.
@nohn thanks for this, this is exactly what it is intended by the mentioned change in https://github.com/home-assistant/core/issues/102937#issuecomment-1783830912 Those the automations should already be broken before 2023.11, but with 2023.11 you get notified about them.
able to go back to 2023.10.3 and the issue has gone away
@Campagne8758 further, please could you verify, if the the supposedly in 2023.11 broken automations are really working again after the rollback?
I did some more digging. I pulled my oldest backup (unfortunately I only keep them for 7 days) and it looks like the "new" device IDs were already present back in core 2023.10.3 based on what I'm seeing in the device registry.
So the change of device IDs happened at some point in the past, but the automations are only now failing due to the PR @mib1185 listed.
So ultimately, at least for my case, this doesn't look like it was caused by this beta - but this beta is bringing the issue to the surface. Which is a good thing, but is probably worth calling out as there will likely be more people that discover they have broken automations after this version.
Is exactly this, because of the PR done now we actually are able to see that a automation is not correct, so personally I think we should see this as a plus. Better just solve the broken automations instead of waiting for a "fix" that actually still not makes your automations be as should be.
Also as already mentioned instead of device triggers just use entity state triggers in case you want to prevent having to make some repairs in case future device ID changes occur. 😀
@Campagne8758 further, please could you verify, if the the supposedly in 2023.11 broken automations are really working again after the rollback?
Yes the automations are back in a working state, one of them turns on the light in the kitchen so I would've imemdaitely noticed if it wasn't working before and it's working again after the rollback. I should mention, the automation were showing as unavailable in 2023.11 but I did not actually check whether they worked anyway or not.
For me, the device IDs remained unchanged which makes sense, as 2023.11 does not change device IDs, just adds some extra checks for device IDs.
@nohn thanks for this, this is exactly what it is intended by the mentioned change in #102937 (comment) Those the automations should already be broken before 2023.11, but with 2023.11 you get notified about them.
That was not the case with me. I realized the problem very soon after upgrading to 2023.11 because an automation that's executed multiple times a day that worked fine until the upgrade stopped working right after the upgrade.