Netatmo webhooks and lights unavailable.

[djfanatix]

The problem

Since the update to 2024.5.x Netatmo webhooks and lights don't work anymore. (they keep marked as offline) They have been working fine for long time.

What version of Home Assistant Core has the issue?

core-2024.5.1

What was the last working version of Home Assistant Core?

core-2024.4.x

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Netatmo

Link to integration documentation on our website

https://www.home-assistant.io/integrations/netatmo/

Diagnostics information

home-assistant_netatmo_2024-05-04T07-12-34.322Z.log

Example YAML snippet

No response

Anything in the logs that might be useful for us?

2024-05-04 09:11:57.403 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py", line 350, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "aiohttp/_http_parser.pyx", line 557, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message:
  Invalid method encountered:

    b'\x16\x03\x03'
      ^

Additional information

I did try:

• reboot HA and OS
• delete and reinstall netatmo integration
• Power off/on netatmo devices
• delete and add webhook again

[home-assistant[bot]]

Hey there @cgtobi, mind taking a look at this issue as it has been labeled with an integration (netatmo) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of netatmo can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign netatmo Removes the current integration label and assignees on the issue, add the integration domain after the command.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

netatmo documentation netatmo source _{^{(message by IssueLinks)}}

[cgtobi]

There are too many errors in that log to make reliable sense of it, especially since many com from custom components. I doubt that the excerpt is related to the Netatmo webhook.

[djfanatix]

Sorry, I put the wrong log, I edited a log and I have replaced it

[propi62]

Same problem

[Alomon31]

Same problem here. Reported on HA forum by other users as well: https://community.home-assistant.io/t/how-to-setup-netatmo-integration-using-webhooks-to-speed-up-device-status-update-response-times/385898/15 and https://community.home-assistant.io/t/netatmo-camera-light-issue-since-2024-05/724560/3

[nrenou-git]

problem seems to be on netatmo side. Dev portal is not woking for more than a week for me. dev Apps stays "desactivated" https://helpcenter.netatmo.com/hc/fr/community/posts/18630944611730-Dev-portal-issues

[Alomon31]

True. Does anyone know a solution to continue using Netatmo integration fully on Home Assistant? Thanks

[djfanatix]

I think once you have the key, you can send local commands to the ip address of your camera.

[crazykiwi2003]

Same problem here - lights unavailable... :-(

[nrenou-git]

netatmo dev website is working again, but i m still not getting any data from the webhooks lights entities unavailable from presences cameras and no data from the welcome camera ( tags detectors etc ... )

log info : service call : unregister webhook 2024-05-15 09:37:59.994 DEBUG (MainThread) [homeassistant.components.netatmo.data_handler] Calls per hour: 71 2024-05-15 09:38:06.168 DEBUG (MainThread) [homeassistant.components.netatmo] Unregister Netatmo webhook (d0...) 2024-05-15 09:38:06.168 INFO (MainThread) [homeassistant.components.netatmo.data_handler] Netatmo webhook unregistered 2024-05-15 09:38:06.291 DEBUG (MainThread) [pyatmo.auth] dropwebhook: <ClientResponse(https://api.netatmo.com/api/dropwebhook) [200 OK]> <CIMultiDictProxy('Server': 'nginx', 'Date': 'Wed, 15 May 2024 07:38:06 GMT', 'Content-Type': 'application/json; charset=utf-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Cache-Control': 'no-cache, must-revalidate', 'Expires': '0', 'X-XSS-Protection': '1; mode=block', 'Access-Control-Allow-Origin': '*', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'X-Powered-By': 'Netatmo', 'Content-Encoding': 'gzip')> 2024-05-15 09:39:01.218 DEBUG (MainThread) [homeassistant.components.netatmo.data_handler] Calls per hour: 71 2024-05-15 09:40:00.288 DEBUG (MainThread) [homeassistant.components.netatmo.data_handler] Calls per hour: 74 2024-05-15 09:40:25.342 DEBUG (MainThread) [homeassistant.components.netatmo] Unregister Netatmo webhook (d0...) 2024-05-15 09:40:25.342 INFO (MainThread) [homeassistant.components.netatmo.data_handler] Netatmo webhook unregistered 2024-05-15 09:40:25.443 DEBUG (MainThread) [pyatmo.auth] The Netatmo API returned b'{"error":{"code":7,"message":"Nothing to drop"}}' (406) 2024-05-15 09:40:25.443 DEBUG (MainThread) [homeassistant.components.netatmo] No webhook to be dropped for d0...

service call : register webhook 2024-05-15 09:42:01.825 DEBUG (MainThread) [homeassistant.components.netatmo.data_handler] Calls per hour: 72 2024-05-15 09:42:02.331 DEBUG (MainThread) [pyatmo.auth] addwebhook: <ClientResponse(https://api.netatmo.com/api/addwebhook) [200 OK]> <CIMultiDictProxy('Server': 'nginx', 'Date': 'Wed, 15 May 2024 07:42:02 GMT', 'Content-Type': 'application/json; charset=utf-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Cache-Control': 'no-cache, must-revalidate', 'Expires': '0', 'X-XSS-Protection': '1; mode=block', 'Access-Control-Allow-Origin': '*', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'X-Powered-By': 'Netatmo', 'Content-Encoding': 'gzip')> 2024-05-15 09:42:02.332 INFO (MainThread) [homeassistant.components.netatmo] Register Netatmo webhook: https://hooks.nabu.casa/gAAA...

[Alomon31]

I've got webhooks working here again, did you try to regenerate your client id+secret? Then delete your integration and your credential information in HA, and set it up again. But my lights are still unavailable after multiple reboots and hours waiting... Any help?

[nrenou-git]

Yes i did all this, still the same, no data from netatmo servers.

[RealGandy]

Same issue here.

Cameras and webhooks are back but light is still showing as unavailable.

As the integration between HA and Netatmo is owned by HA, can someone do some investigations into what request is made and what response is expected for the light to be active again?

Or share info with consumers so we can approach Netatmo directly with evidence of what is going wrong.

Cheers

[Alomon31]

According to Netatmo dev team (https://helpcenter.netatmo.com/hc/fr/community/posts/18630944611730/comments/18981824506258), the issue is now on HA side. @cgtobi , would you mind having a look at it? Thank you very much.

[RealGandy]

I was going to post this to help wth working out what the problem was, but having just run through this sequence, the light is now no longer showing as unavailable... and seems to be working as expected for me.

======================================================= Here are some entries in log with netatmo in debug mode:

I restarted the integration - and you can see a successful webhook activation. I went outside and the camera spotted me.

followed by me turning the light on, off and back to auto via the netatmo app, you can see incoming webhooks from netatmo showing those activities.

[nrenou-git]

I was finally able to get webhooks working again. The problem came from my configuration but it was not obvious, because I had to configure the client ID and secret in a homeassistant menu that I had never had to use. Previously my netatmo configuration was declared in configuration.yaml then as home assistant was updated the installer asked me to remove the identification information from configuration.yaml. I never had to add my credentials in the new "application credentials" menu. On the other hand, I don't understand that when registering the webhook via the netatmo.register_webhook service I didn't get an error message in the log? this would have allowed me to understand that something was missing in my configuration. The only message I got was "<ClientResponse(https://api.netatmo.com/api/addwebhook) [200 OK]>"

[djfanatix]

Indeed many thanks. I assume the problem is for users who don't use the Nabu Casa cloudservice. Steps to make it working:

• add app credentials (integrations page)
• delete Netatmo integration
• add Netatmo again, don't choose 'home assistant cloud', but your own credentials
• following the links

[Princeless74]

Hello, It works and the light works great, but no more events are triggered when someone or a car is detected.

If you run in debug mode, the webhook seems to work since the event detected is received on the log, but it doesn't trigger anything in my automations (that were working before and also one more I created just for testing purpose)

[oywino]

My attempt to run the service Netamo:Register webhook only results in this error message:

Failed to call service netatmo.register_webhook. Unknown error

I see the following on the log:

homeass-homeass-1  | 2024-08-30 21:05:15.417 ERROR (MainThread) [homeassistant.helpers.script.websocket_api_script] websocket_api script: Error executing script. Unexpected error for call_service at pos 1: Handler is already defined!
homeass-homeass-1  | Traceback (most recent call last):
homeass-homeass-1  |   File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 525, in _async_step
homeass-homeass-1  |     await getattr(self, handler)()
homeass-homeass-1  |   File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 764, in _async_call_service_step
homeass-homeass-1  |     response_data = await self._async_run_long_action(
homeass-homeass-1  |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
homeass-homeass-1  |   File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 727, in _async_run_long_action
homeass-homeass-1  |     return await long_task
homeass-homeass-1  |            ^^^^^^^^^^^^^^^
homeass-homeass-1  |   File "/usr/src/homeassistant/homeassistant/core.py", line 2763, in async_call
homeass-homeass-1  |     response_data = await coro
homeass-homeass-1  |                     ^^^^^^^^^^
homeass-homeass-1  |   File "/usr/src/homeassistant/homeassistant/core.py", line 2806, in _execute_service
homeass-homeass-1  |     return await target(service_call)
homeass-homeass-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
homeass-homeass-1  |   File "/usr/src/homeassistant/homeassistant/components/netatmo/__init__.py", line 157, in register_webhook
homeass-homeass-1  |     webhook_register(
homeass-homeass-1  |   File "/usr/src/homeassistant/homeassistant/components/webhook/__init__.py", line 54, in async_register
homeass-homeass-1  |     raise ValueError("Handler is already defined!")
homeass-homeass-1  | ValueError: Handler is already defined!

The Netatmo Integration as such works and I have all entities available. I have established correct Application credentials and re-installed the integration, but to no avail; Webhooks persistently refuse to work. I run HA Core Docker 2024.8.3

[radokristof]

I also have some problems with custom webhook and the Presence Light switch is unavailable.

I receive this in the log:

2024-09-09 22:02:47.924 ERROR (MainThread) [homeassistant.components.netatmo] Error during webhook registration - ("400 - Bad request - Invalid url parameter (21) when accessing 'https://api.netatmo.com/api/addwebhook'",)

The home assistant instance is reachable publicly through a web proxy (port 443)

[Princeless74]

@cgtobi Since my last message, I still have issues with triggering events. I tried to create a new App on dev.netatmo, and also deleted my cameras and recreated them on a new email account. Still the same : Webhooks are ok, handling of light is working, but no event like human detected or car detected are triggered in HA.

From my logs, I could see the event infos received, but it seems they are received with wrong user_id / snapshot_id / home_id. Is it maybe due that at a point (with my old account, not wit hthe new one) I clicked on reset keys and generate token ?

2024-08-07 15:05:49.561 DEBUG (MainThread) [homeassistant.components.netatmo.data_handler] Calls per hour: 179 2024-08-07 15:06:32.153 DEBUG (MainThread) [homeassistant.components.netatmo.webhook] Got webhook data: { 'user_id': '63bxxxxxxx', 'snapshot_id': '66b3xxxxxxx', 'snapshot_key': '39d8daaxxxxxxx', 'snapshot_url': 'xxxxxxx', 'vignette_id': '66b3xxxxxxx', 'vignette_key': 'b19a697xxxxxxx', 'vignette_url': 'xxxxxxx', 'event_type': 'human', 'camera_id': '70:ee:50:30:15:ac', 'device_id': '70:ee:50:30:15:ac', 'home_id': '63d8xxxxxxx', 'home_name': 'Hery', 'event_id': '66b3xxxxxxx', 'subevent_id': '65eaexxxxxxx', 'message': 'Personne détectée', 'push_type': 'NOC-human' } 2024-08-07 15:06:32.154 DEBUG (MainThread) [homeassistant.components.netatmo.webhook] human: {'user_id': '63bxxxxxxx', 'snapshot_id': '66bxxxxxxx', 'snapshot_key': '39d8daa24xxxxxxx', 'snapshot_url': xxxxxxx', 'vignette_id': '66b3xxxxxxx', 'vignette_key': 'b19axxxxxxx', 'vignette_url': 'xxxxxxx', 'event_type': 'human', 'camera_id': '70:xxxxxxx', 'device_id': '70:xxxxxxx', 'home_id': '63dxxxxxxx', 'home_name': 'Hery', 'event_id': '66b3xxxxxxx', 'subevent_id': '65eaexxxxxxx', 'message': 'Personne détectée', 'push_type': 'NOC-human'}

2024-08-07 15:06:50.131 DEBUG (MainThread) [homeassistant.components.netatmo.data_handler] Calls per hour: 164

Thanks for your help :)

[oywino]

What do you mean by "Webhooks are ok" ? Can you listen for netatmo_event in the developer section and actually se something there?

[Princeless74]

Hi @oywino, By "webhooks are ok" I mean it is correctly setup as described on the guide. As the light works, it means that the webhook correctly communicate. Moreover, we can see in HA log that it receive the events, so webhook communication between netatmo and HA is ok.

If i suscribe the events, it correctly receive the detection event and I can see it correctly.

My hint is that the netatmo send data with wrong ID somehow...

[oywino]

I still don't understand. If webhook is correctly set up and you can see in the HA log that events are received, and you can subscribe to see Netatmo events, then what more do you need? As you describe, everything seems to work perfectly for you.

[Princeless74]

Hi, I know, but if I create on both cameras an automation "when a person is detected", thing is never triggered. It used to work for one year and it stopped working from may when netatmo integration had some issues (and from this date also I reseted the key on dev.netatmo portal).

I am pretty sure that it doesn't trigger the automation because the event received contains wrong IDs and then, because of this, it doesn't trigger the automation.

[oywino]

If your Netatmo camera's are integrated using the Netatmo Integration, then i doubt that you have presence detection sensor. As far as I know, this integration does not support such entities. But if you have your cameras integrated using the HomeKit Integration, then you will have such sensors. However, under normal circumstances you cannot have the same camera integrated in two integrations simultaneously (but it has been known to happen). If you want to use event detection to trigger based on the Netatmo Integration via netatmo_event then you must use the event platform as your trigger. Here's an automation that just reads a text out loud on a mobile phone speaker whenever a Netatmo event happens (any Netatmo event):

alias: "Test: Netatmo Event"
description: ""
trigger:
  - platform: event
    event_type: netatmo_event
condition: []
action:
  - metadata: {}
    data:
      message: TTS
      data:
        tts_text: Netamo event detected!
        media_stream: alarm_stream_max
        ttl: 0
        priority: high
    action: notify.mobile_app_sm_g996b
mode: single

Just replace the action entity with the correct one for your mobile phone (assuming you have integrated your mobile phone with Home Assistant).

If you had motion detection working earlier, it must have been because your HomeKit integration was live and working with your Netatmo cameras. That I think was removed (bug-fixed) during one of the more resent updates (so you can no longer have a Netatmo camera device integrated twice).

[Princeless74]

Yes, my Netatmo cameras are integrated using Netatmo integration. If you create an automation, you can trigger your automation based on the camera detection car/human/animal/other. It used to work fine before for months :)

Actually, my issue is that when netatmo_event, theses automations are not trigered anymore.

I will try to create manually something like you mentioned, but in parallel, I think there is a bug somethere ;)

Is there a way to compare reveiced IDs with my cameras IDs ?

[oywino]

I see what you do. You could (should) use entity instead of device in all your automations because devices use id as a reference while entities use names. If you delete and reinstall a device, it will get a new device_id and all your automations has to be rewritten. While an entity maintains its name. Try this instead:

trigger:
  - platform: state
    entity_id:
      - camera.camera01
    attribute: motion_detection
    to: "on"
condition: []
action: []
mode: single

[Princeless74]

I would agree with you, but using entity means, only trigger when a motion is detected : I need to distinguish car/human vs animal/other and that's why I need to use device.

The idea behind is to send me alert is my home alarm is ON and a car/human is detected, without beoing annoyed by animals like cats ;)

[oywino]

Well, that at least re-create your automation so that you get the correct ID updated. Also, carefully examine the trace before you conclude that it doesn't trigger. Don't trust the last_triggered variable.