Adds adopt/unadopt flows for UniFi Protect devices

[AngellusMortis]

Proposed change

Adds Adopt button for unadopted Protect devices as well as Unadopt button for adopted Protect devices,

Unadopt button is disabled by default since it is a destructive action with no confirmation. Also, the unadopt flow is kind of bad right now since the device itself is not removed from Home Assistant when it is unadopted. However, automatically removing unadopted devices is planned for the next PR. The unadopt button was really easy to add and helpful for testing the adopt button which is why it was included in this PR.

Type of change

• [ ] Dependency upgrade
• [ ] Bugfix (non-breaking change which fixes an issue)
• [ ] New integration (thank you!)
• [X] New feature (which adds functionality to an existing integration)
• [ ] Deprecation (breaking change to happen in the future)
• [ ] Breaking change (fix/feature causing existing functionality to break)
• [ ] Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• [ ] The code change is tested and works locally.
• [ ] Local tests pass. Your PR cannot be merged unless tests pass
• [ ] There is no commented out code in this PR.
• [ ] I have followed the development checklist
• [ ] The code has been formatted using Black (black --fast homeassistant tests)
• [ ] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• [ ] Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• [ ] The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• [ ] New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• [ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
• [ ] Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• [ ] No score or internal
• [ ] 🥈 Silver
• [ ] 🥇 Gold
• [ ] 🏆 Platinum

To help with the load of incoming pull requests:

• [ ] I have reviewed two other open pull requests in this repository.

[probot-home-assistant[bot]]

Hey there @briis, @bdraco, mind taking a look at this pull request as it has been labeled with an integration (unifiprotect) you are listed as a code owner for? Thanks! _{^{(message by CodeOwnersMention)}}

[bdraco]

Thinking about this some more, I think it would be a nicer workflow to be able to create discovery flows for each discovered camera. That would require some core changes though to make it possible. Maybe something for the future.

In the mean time I need to find a camera I can reset

[AngellusMortis]

Yeah, I was thinking that too. The other integrations that allow you to add devices (ZHA, ZWaveJS, Insteon) all have their own discovery flows. I think it would be nice for there to some kind of unified discovery flow for integrations.

[bdraco]

The adopt button appears but its unavailable

[bdraco]

Restarting made the button available.

It seems like its only a problem if the integration is added after the device is available to adopt?

[AngellusMortis]

It seems like its only a problem if the integration is added after the device is available to adopt?

I will look into it in the morning. I am guessing it is because there is a setting missing that allows the device to be loaded into the initial bootstrap on start. I want to update the attr_available logic for that button as well since it is different.

[AngellusMortis]

I was not able to reproduce the behavior with my AI Theta. I plugged the device in, let it shows up as ready for adoption and then started HA. The button showed up correctly and everything.

I did update the attr_available for that button though so it uses the same logic it uses to create the button so hopefully it should always be online now if it is adoptable. Also found a bug introduce by the privacy mode PR that was a oneline fix.

[bdraco]

Retesting. Button worked 👍

But I got an exception when I used it and the camera is showing unavailable

2022-08-25 13:34:47.935 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new button.unifiprotect entity: button.g4_bullet_adopt_device

2022-08-25 13:35:06.896 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new binary_sensor.unifiprotect entity: binary_sensor.g4_bullet_is_dark
2022-08-25 13:35:06.898 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new binary_sensor.unifiprotect entity: binary_sensor.g4_bullet_motion
2022-08-25 13:35:06.899 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new button.unifiprotect entity: button.g4_bullet_reboot_device
2022-08-25 13:35:06.899 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new button.unifiprotect entity: button.g4_bullet_unadopt_device
2022-08-25 13:35:06.900 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new number.unifiprotect entity: number.g4_bullet_wide_dynamic_range
2022-08-25 13:35:06.911 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new select.unifiprotect entity: select.g4_bullet_recording_mode
2022-08-25 13:35:06.911 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.unifiprotect entity: sensor.g4_bullet_oldest_recording
2022-08-25 13:35:06.911 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.unifiprotect entity: sensor.g4_bullet_storage_used
2022-08-25 13:35:06.912 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.unifiprotect entity: sensor.g4_bullet_disk_write_rate
2022-08-25 13:35:06.912 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.unifiprotect entity: sensor.g4_bullet_received_data
2022-08-25 13:35:06.912 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.unifiprotect entity: sensor.g4_bullet_transferred_data
2022-08-25 13:35:06.912 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.unifiprotect entity: sensor.g4_bullet_uptime
2022-08-25 13:35:06.912 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.unifiprotect entity: sensor.g4_bullet_last_motion_detected
2022-08-25 13:35:06.912 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new switch.unifiprotect entity: switch.g4_bullet_ssh_enabled
2022-08-25 13:35:06.912 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new switch.unifiprotect entity: switch.g4_bullet_overlay_show_name
2022-08-25 13:35:06.912 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new switch.unifiprotect entity: switch.g4_bullet_overlay_show_date
2022-08-25 13:35:06.912 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new switch.unifiprotect entity: switch.g4_bullet_overlay_show_logo
2022-08-25 13:35:06.912 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new switch.unifiprotect entity: switch.g4_bullet_overlay_show_bitrate
2022-08-25 13:35:06.913 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new switch.unifiprotect entity: switch.g4_bullet_detections_motion
2022-08-25 13:35:06.913 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new switch.unifiprotect entity: switch.g4_bullet_privacy_mode
2022-08-25 13:35:09.414 ERROR (MainThread) [pyunifiprotect.websocket] Error processing websocket message
Traceback (most recent call last):
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/websocket.py", line 69, in _process_message
    sub(msg)
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/api.py", line 637, in _process_ws_message
    processed_message = self.bootstrap.process_ws_packet(
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/bootstrap.py", line 468, in process_ws_packet
    return self._process_device_update(packet, action, data, ignore_stats)
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/bootstrap.py", line 394, in _process_device_update
    obj = obj.update_from_dict(deepcopy(data))
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/devices.py", line 858, in update_from_dict
    return super().update_from_dict(data)
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/base.py", line 473, in update_from_dict
    setattr(self, key, unifi_obj.update_from_dict(data.pop(key)))
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/base.py", line 473, in update_from_dict
    setattr(self, key, unifi_obj.update_from_dict(data.pop(key)))
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/base.py", line 469, in update_from_dict
    data_set = set(data)
TypeError: 'NoneType' object is not iterable
2022-08-25 13:35:11.022 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new camera.unifiprotect entity: camera.g4_bullet_high

[bdraco]

Remains unavailable after reload, but more entities get created

2022-08-25 13:36:36.436 INFO (MainThread) [pyunifiprotect.api] Websocket re-connected successfully
2022-08-25 13:36:37.182 INFO (MainThread) [homeassistant.components.binary_sensor] Setting up binary_sensor.unifiprotect
2022-08-25 13:36:37.182 INFO (MainThread) [homeassistant.components.button] Setting up button.unifiprotect
2022-08-25 13:36:37.182 INFO (MainThread) [homeassistant.components.camera] Setting up camera.unifiprotect
2022-08-25 13:36:37.182 INFO (MainThread) [homeassistant.components.light] Setting up light.unifiprotect
2022-08-25 13:36:37.183 INFO (MainThread) [homeassistant.components.lock] Setting up lock.unifiprotect
2022-08-25 13:36:37.183 INFO (MainThread) [homeassistant.components.media_player] Setting up media_player.unifiprotect
2022-08-25 13:36:37.183 INFO (MainThread) [homeassistant.components.number] Setting up number.unifiprotect
2022-08-25 13:36:37.183 INFO (MainThread) [homeassistant.components.select] Setting up select.unifiprotect
2022-08-25 13:36:37.183 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.unifiprotect
2022-08-25 13:36:37.183 INFO (MainThread) [homeassistant.components.switch] Setting up switch.unifiprotect
2022-08-25 13:36:37.200 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new number.unifiprotect entity: number.g4_bullet_microphone_level
2022-08-25 13:36:37.203 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new select.unifiprotect entity: select.g4_bullet_infrared_mode
2022-08-25 13:36:37.215 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.unifiprotect entity: sensor.g4_bullet_detected_object
2022-08-25 13:36:37.220 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new switch.unifiprotect entity: switch.g4_bullet_hdr_mode
2022-08-25 13:36:37.220 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new switch.unifiprotect entity: switch.g4_bullet_high_fps
2022-08-25 13:36:37.221 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new switch.unifiprotect entity: switch.g4_bullet_detections_person
2022-08-25 13:36:37.221 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new switch.unifiprotect entity: switch.g4_bullet_detections_vehicle

[bdraco]

Reloaded again and everything worked. So likely a race somewhere

[AngellusMortis]

It is probably all from the initial error you got. Is that consistency reproducible (unadopt the device, delete the device from Home Assistant, and do it again)?

[bdraco]

Repoed 2 more times

Same exception

2022-08-25 13:44:56.681 ERROR (MainThread) [pyunifiprotect.websocket] Error processing websocket message
Traceback (most recent call last):
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/websocket.py", line 69, in _process_message
    sub(msg)
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/api.py", line 637, in _process_ws_message
    processed_message = self.bootstrap.process_ws_packet(
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/bootstrap.py", line 468, in process_ws_packet
    return self._process_device_update(packet, action, data, ignore_stats)
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/bootstrap.py", line 394, in _process_device_update
    obj = obj.update_from_dict(deepcopy(data))
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/devices.py", line 858, in update_from_dict
    return super().update_from_dict(data)
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/base.py", line 473, in update_from_dict
    setattr(self, key, unifi_obj.update_from_dict(data.pop(key)))
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/base.py", line 473, in update_from_dict
    setattr(self, key, unifi_obj.update_from_dict(data.pop(key)))
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/base.py", line 469, in update_from_dict
    data_set = set(data)
TypeError: 'NoneType' object is not iterable

[AngellusMortis]

Made a fix: https://github.com/AngellusMortis/pyunifiprotect/pull/221

[AngellusMortis]

Pulled in the dep update into this PR. I can split it back out if it works after testing is finished.

[bdraco]

Pulled in the dep update into this PR. I can split it back out if it works after testing is finished.

Will retest shortly

[bdraco]

2022-08-25 14:50:07.153 ERROR (MainThread) [pyunifiprotect.websocket] Error processing websocket message
Traceback (most recent call last):
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/websocket.py", line 69, in _process_message
    sub(msg)
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/api.py", line 637, in _process_ws_message
    processed_message = self.bootstrap.process_ws_packet(
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/bootstrap.py", line 470, in process_ws_packet
    return self._process_device_update(packet, action, data, ignore_stats)
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/bootstrap.py", line 396, in _process_device_update
    obj = obj.update_from_dict(deepcopy(data))
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/devices.py", line 858, in update_from_dict
    return super().update_from_dict(data)
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/base.py", line 473, in update_from_dict
    setattr(self, key, unifi_obj.update_from_dict(data.pop(key)))
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/base.py", line 473, in update_from_dict
    setattr(self, key, unifi_obj.update_from_dict(data.pop(key)))
  File "/Users/bdraco/home-assistant/venv/lib/python3.10/site-packages/pyunifiprotect/data/base.py", line 469, in update_from_dict
    data_set = set(data)
TypeError: 'NoneType' object is not iterable

Same trace with this commit

commit e9fa4871b5952fcc0071e3aa10b14208a883dc68 (ufp-adopt)
Author: Christopher Bailey <[email protected]>
Date:   Thu Aug 25 15:22:01 2022 -0400

    Updates pyufp

diff --git a/homeassistant/components/unifiprotect/manifest.json b/homeassistant/components/unifiprotect/manifest.json
index 2d3b664382..d00e788f9b 100644
--- a/homeassistant/components/unifiprotect/manifest.json
+++ b/homeassistant/components/unifiprotect/manifest.json
@@ -3,7 +3,7 @@
   "name": "UniFi Protect",
   "config_flow": true,
   "documentation": "https://www.home-assistant.io/integrations/unifiprotect",
-  "requirements": ["pyunifiprotect==4.1.5", "unifi-discovery==1.1.5"],
+  "requirements": ["pyunifiprotect==4.1.6", "unifi-discovery==1.1.5"],
   "dependencies": ["http"],
   "codeowners": ["@briis", "@AngellusMortis", "@bdraco"],
   "quality_scale": "platinum",
diff --git a/requirements_all.txt b/requirements_all.txt
index a25550c762..6b21ed3d7f 100644
--- a/requirements_all.txt
+++ b/requirements_all.txt
@@ -2019,7 +2019,7 @@ pytrafikverket==0.2.0.1
 pyudev==0.23.2
 
 # homeassistant.components.unifiprotect
-pyunifiprotect==4.1.5
+pyunifiprotect==4.1.6