core
core copied to clipboard
home-assistant
UniFi Network Integration Failing after Cloud Key Firmware Update
The problem
I just updated my UniFi Cloud Key Gen 2 firmware from 1.1.x to 2.1.11 - referring to the actual device firmware, not the network application specific firmware. This update made a key configuration change:
- Previously the core device microsite would be reached at https://ubnt.example.co:443 (my domain, local DNS) while the network app, including api, could be reached at https://ubnt.example.co:8443
- Now the sites are merged; the core microsite is https://ubnt.example.co:443, while the network site is https://ubnt.example.co:443/network/...
So Home Assistant is failing to find the new API location because it automatically looks at ubnt.example.co/api, but now probably needs to look at ubnt.example.co/network/api or something similar.
I couldn't find any setting I could change to correct this, but it seems like the integration would need to check both directory structures for the api location.
What version of Home Assistant Core has the issue?
2022.8.7
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant Core
Integration causing the issue
UniFi Network
Link to integration documentation on our website
https://www.home-assistant.io/integrations/unifi
Diagnostics information
No response
Example YAML snippet
No response
Anything in the logs that might be useful for us?
No response
Additional information
No response
![probot-home-assistant[bot] avatar](https://avatars.githubusercontent.com/in/36093?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hey there @kane610, mind taking a look at this issue as it has been labeled with an integration (unifi) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)
Could you enable debug logging for the unifi integration? Instructions are listed in the integration documentation.
Enabled the debug logging. Is this the out you are locking for.
2022-09-16 11:18:59.294 DEBUG (MainThread) [aiounifi.controller] 200 application/json <ClientResponse(https://IP adress Cloudkey:443/proxy/network/api/s/default/rest/user) [200 OK]> <CIMultiDictProxy('Vary': 'accept-encoding', 'X-DNS-Prefetch-Control': 'off', 'x-frame-options': 'DENY', 'Strict-Transport-Security': 'max-age=15552000; includeSubDomains', 'X-Download-Options': 'noopen', 'X-Content-Type-Options': 'nosniff', 'X-XSS-Protection': '1; mode=block', 'Accept-Ranges': 'bytes', 'Content-Encoding': 'gzip', 'Content-Type': 'application/json;charset=UTF-8', 'Date': 'Fri, 16 Sep 2022 09:18:59 GMT', 'X-CSRF-Token': '0a7f8490-6e25-4b4e-b7ae-e6163711c13a', 'Set-Cookie': 'TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJkNDljMTQzOC0yOGM1LTRjODEtOGI1ZC1lYjgzYjU0MzcwOWYiLCJjc3JmVG9rZW4iOiIwYTdmODQ5MC02ZTI1LTRiNGUtYjdhZS1lNjE2MzcxMWMxM2EiLCJqdGkiOiJkZTJlZjViZC0xYTljLTQ1ZmItOTNjZS1hY2M2YzY1ODdjZmMiLCJwYXNzd29yZFJldmlzaW9uIjoxNjYyNzEwMjgzLCJpYXQiOjE2NjMzMTk5MzksImV4cCI6MTY2MzMyMzUzOX0.kJTqq0r2AhFoCsZwSA5fbCgO1G45qaJJehdrw8J9jPg; path=/; samesite=none; secure; httponly', 'Connection': 'keep-alive', 'Transfer-Encoding': 'chunked')>
2022-09-16 11:18:59.296 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry Default for unifi Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 357, in async_setup result = await component.async_setup_entry(hass, self) File "/usr/src/homeassistant/homeassistant/components/unifi/init.py", line 40, in async_setup_entry await controller.initialize() File "/usr/src/homeassistant/homeassistant/components/unifi/controller.py", line 310, in initialize await self.api.initialize() File "/usr/local/lib/python3.10/site-packages/aiounifi/controller.py", line 161, in initialize await self.clients_all.update() File "/usr/local/lib/python3.10/site-packages/aiounifi/interfaces/api.py", line 36, in update self.process_raw(raw) File "/usr/local/lib/python3.10/site-packages/aiounifi/interfaces/api.py", line 44, in process_raw key = raw_item[self.KEY] KeyError: 'mac'
I've got the funny feeling Ubiquity changed their API side in 7.3.x causing aiounifi/unifi to stop working.
FYI @Kane610 I've tried your latest aiounifi from Github, no success. Error messages as per above remained (albeit different line numbers of course).
If you want me to test anything, love to help but my Python skills are basically non-existent :(
I've got the funny feeling Ubiquity changed their API side in 7.3.x causing aiounifi/unifi to stop working.
FYI @Kane610 I've tried your latest aiounifi from Github, no success. Error messages as per above remained (albeit different line numbers of course).
If you want me to test anything, love to help but my Python skills are basically non-existent :(
Well the changes in the library are refactorings that should keep working so thanks for verifying that :)
Do you get the 404 error or the Mac error shared by @bwestrek ?
I can't really debug this myself as I run a udm pro.
What is needed is as OP looked and that is to do wiretracing on the network app in a browser and figure out the new paths.
If you can run the library standalone you can really cut down on iteration time and try out changing the api paths in the lib (I can coach on this)
@Kane610 I get the Mac error. I do not own a Cloud Key but am running the Unifi controller locally. I'd be happy to test and I can run libraries standalone, but would need some guidance. Thanks!
I've gotten a bit further: aiounifi/controller.py line 161 (166 in my updated version) is the culprit. When I disable this line, all works fine: '# await self.clients_all.update()
I'm not sure what this function is used for as the file "aiounifi/interfaces/clients_all.py seems to run the API call but then doesn't seem to do much (unless I'm completely wrong, just let me know). The API call in clients_all.py is https://controller_hostname:8443/api/s/default/rest/user which is valid. The API call in clients.py is https://controller_hostname:8443/api/s/default/stat/sta which is also valid.
I guess there is an issue with clients_all.py interpreting the returned json wrongly? Would a quick solution be to remove line 161 or will anything else be broken then?
Ok, final update for today (no promises though 😀 ).
I've compared the json outputs of both /rest/user and /stat/sta.
All entities in /rest/user are also available in /stat/sta (e.g. _id, mac, site_id, is_guest, etc), so far so good.
However, I noticed 1 entry in /rest/user (called by clients_all.py) which is different:
{ "_id": "6319f98e85d118fbexxxxxxx", "local_dns_record_enabled": false, "site_id": "5ba53008d5fb1d03dxxxxxxx" },
As this entry is missing mac, I guess loading this entry fails causing the error we've seen, or am I wrong? This is the only entry with a missing mac address. If this is the issue, would it be easy to add a check for this failure somewhere (here comes my lack of Python skills in play).
Slightly related, as mentioned the integration loads fine when line 161 is disabled. However, the following error now shows up. I guess Unifi added a new Alert category causing this message:
2022-09-20 14:18:20.950 WARNING (MainThread) [aiounifi.models.message] Unsupported message key alert
2022-09-20 14:18:20.955 DEBUG (MainThread) [aiounifi.websocket] {"meta":{"rc":"ok","message":"alert"},"data":[{"category":"SYSTEM","id":"6329af8c95c4e7000xxxxxxx","key":"ADMIN_ACCESS","message":"homeassistant accessed the UniFi Network web application.","message_raw":"{ADMIN} accessed the UniFi Network {PLATFORM} application.","parameters":{"ADMIN":{"id":"5c38b71fd5fb1d4dexxxxxxx","name":"homeassistant"},"PLATFORM":{"name":"web"}},"replace_alerts":[],"severity":"INFO","show_on_dashboard":false,"status":"NEW","target":"ADMIN","timestamp":1663676300928,"title_raw":"Admin Accessed Console","type":"ADMIN_ACCESS"}]}
Ok, final update for today (no promises though 😀 ). I've compared the json outputs of both /rest/user and /stat/sta. All entities in /rest/user are also available in /stat/sta (e.g. _id, mac, site_id, is_guest, etc), so far so good. However, I noticed 1 entry in /rest/user (called by clients_all.py) which is different:
{ "_id": "6319f98e85d118fbexxxxxxx", "local_dns_record_enabled": false, "site_id": "5ba53008d5fb1d03dxxxxxxx" },As this entry is missing mac, I guess loading this entry fails causing the error we've seen, or am I wrong? This is the only entry with a missing mac address. If this is the issue, would it be easy to add a check for this failure somewhere (here comes my lack of Python skills in play).
Slightly related, as mentioned the integration loads fine when line 161 is disabled. However, the following error now shows up. I guess Unifi added a new Alert category causing this message:
2022-09-20 14:18:20.950 WARNING (MainThread) [aiounifi.models.message] Unsupported message key alert 2022-09-20 14:18:20.955 DEBUG (MainThread) [aiounifi.websocket] {"meta":{"rc":"ok","message":"alert"},"data":[{"category":"SYSTEM","id":"6329af8c95c4e7000xxxxxxx","key":"ADMIN_ACCESS","message":"homeassistant accessed the UniFi Network web application.","message_raw":"{ADMIN} accessed the UniFi Network {PLATFORM} application.","parameters":{"ADMIN":{"id":"5c38b71fd5fb1d4dexxxxxxx","name":"homeassistant"},"PLATFORM":{"name":"web"}},"replace_alerts":[],"severity":"INFO","show_on_dashboard":false,"status":"NEW","target":"ADMIN","timestamp":1663676300928,"title_raw":"Admin Accessed Console","type":"ADMIN_ACCESS"}]}
Sweet! Good job!
Its a simple fix, unfortunately I have done some reworks of the library so it might warrant waiting for the next release to bring it in just to be safe.
Ok, final update for today (no promises though 😀 ). I've compared the json outputs of both /rest/user and /stat/sta. All entities in /rest/user are also available in /stat/sta (e.g. _id, mac, site_id, is_guest, etc), so far so good. However, I noticed 1 entry in /rest/user (called by clients_all.py) which is different:
{ "_id": "6319f98e85d118fbexxxxxxx", "local_dns_record_enabled": false, "site_id": "5ba53008d5fb1d03dxxxxxxx" },As this entry is missing mac, I guess loading this entry fails causing the error we've seen, or am I wrong? This is the only entry with a missing mac address. If this is the issue, would it be easy to add a check for this failure somewhere (here comes my lack of Python skills in play).
Slightly related, as mentioned the integration loads fine when line 161 is disabled. However, the following error now shows up. I guess Unifi added a new Alert category causing this message:
2022-09-20 14:18:20.950 WARNING (MainThread) [aiounifi.models.message] Unsupported message key alert 2022-09-20 14:18:20.955 DEBUG (MainThread) [aiounifi.websocket] {"meta":{"rc":"ok","message":"alert"},"data":[{"category":"SYSTEM","id":"6329af8c95c4e7000xxxxxxx","key":"ADMIN_ACCESS","message":"homeassistant accessed the UniFi Network web application.","message_raw":"{ADMIN} accessed the UniFi Network {PLATFORM} application.","parameters":{"ADMIN":{"id":"5c38b71fd5fb1d4dexxxxxxx","name":"homeassistant"},"PLATFORM":{"name":"web"}},"replace_alerts":[],"severity":"INFO","show_on_dashboard":false,"status":"NEW","target":"ADMIN","timestamp":1663676300928,"title_raw":"Admin Accessed Console","type":"ADMIN_ACCESS"}]}
Since you're trying out the library you could update it to latest master and see if that resolves your issue :)
Initial test: Works like a charm! I've loaded the latest master and reloaded Home Assistant, and the integration loads and all seems to be fine. I will be checking in over the weekend to see if all remains working fine.
Would you like me to open a new Issue under you project for the missing Alert category (see the last 2 lines in my previous message)? This is still present, though just a warning and not impacting functionality whatsoever.
No as long as I have something to reference in the PR its enough. Thanks for reporting!
Try it out why don't ya! :)