Allow GET in webhook triggers

[swiergot]

Proposed change

Accept HTTP GET method in webhook triggers. So far only POST, PUT and HEAD have been allowed, but many devices which can be configured to send HTTP commands support only GET.

Type of change

• [ ] Dependency upgrade
• [ ] Bugfix (non-breaking change which fixes an issue)
• [ ] New integration (thank you!)
• [x] New feature (which adds functionality to an existing integration)
• [ ] Breaking change (fix/feature causing existing functionality to break)
• [ ] Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• [x] The code change is tested and works locally.
• [x] Local tests pass. Your PR cannot be merged unless tests pass
• [x] There is no commented out code in this PR.
• [x] I have followed the development checklist
• [x] The code has been formatted using Black (black --fast homeassistant tests)
• [x] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• [ ] Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• [ ] The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• [ ] New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• [ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
• [ ] Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• [ ] No score or internal
• [ ] 🥈 Silver
• [ ] 🥇 Gold
• [ ] 🏆 Platinum

To help with the load of incoming pull requests:

• [ ] I have reviewed two other open pull requests in this repository.

[homeassistant]

Hi @swiergot,

It seems you haven't yet signed a CLA. Please do so here.

Once you do that we will be able to review and accept this pull request.

Thanks!

[MartinHjelmare]

See these rejected PRs for history in this matter: https://github.com/home-assistant/core/pull/17782 https://github.com/home-assistant/core/pull/19648

[frenck]

I think we should maybe consider allowing for this, for 2 reasons:

• As said in this and previous PRs, some devices may only support GET requests
• Webhooks can be used to trigger "anything", and doesn't have to provide information even. I can see possibilities to use simple GET requests to trigger automation.

[esev]

If this PR is to apply to all webhooks, is it possible to add CSRF protection on these URLs?

[davet2001]

+1 from me. QR codes and RFID tag URLs could then be used to trigger automations without apps.

[davet2001]

@esev what type of CSRF protection did you have in mind? I don't think any of the use cases wanting this can set header tokens.

[emufan]

+1

[esev]

@esev what type of CSRF protection did you have in mind? I don't think any of the use cases wanting this can set header tokens.

I agree. And I don't see any good solution except for using long random URLs that aren't shared publicly.

Just to clarify. I do see the value in this PR! It makes some integrations a lot simpler. My goal isn't to stop the PR. I just want to raise awareness of the issues. As long as users are aware of the abuse vector, they can take steps to avoid it.

I just fear folks will use this without considering the consequences. http://homeassistant.local:8123/api/webhook/ is a fairly common URL that will work on many people's HA instances. These are unauthenticated URLs. It doesn't take much to abuse this if folks aren't always generating secret random webhook IDs, or if they print the URLs on QR codes/NFC stickers. Any web page or email that contains html similar to the following will be enough. It doesn't matter where the html is hosted; all that matters is where the user's browser is running.

<html>
  <body>
    <img src="http://homeassistant.local:8123/api/webhook/Unlock_Door" />
    <img src="http://homeassistant.local:8123/api/webhook/Open_Garage" />
  </body>
</html>

Can a blueprint contain a webhook trigger? If so that might be another abuse vector to consider (blueprints with common webhook names, or attacker controlled blueprints). Especially with the easy-to-click "Add Blueprint" buttons.

That said, AFAICT the CSRF issue is already present with webhooks as the CORS headers already allow this (cors_allowed = True). This PR just makes it easier to abuse directly from any static email/web page; no javascript required.

This might get safer in the future though. See https://wicg.github.io/private-network-access/. I believe Chrome intends to support this (https://developer.chrome.com/blog/private-network-access-update/).

[emufan]

is a fairly common URL that will work on many people's HA instances

But you are aware that everything you wrote is the same for POST webhooks?

This request is to get GET and not to remove webhooks. 😉

[davet2001]

Unauthenticated webhooks that do things are a security hole. But sometimes necessary. I suggest:

• Off by default
• Red warning text in docs next to turning them on, with advice that they should not be used for destructive actions or anything that might affect safety/security.
• Triggered by local network only (I assume that webhooks are already blocked for anything internet facing, but if not they should be).
• (more advanced) Whitelist IPs (/mac addresses/user agent strings?) that are allowed to trigger unauthenticated webhooks, log in HA when something is blocked.

[emufan]

Why are we discussing webhooks in this PR? This is completely not related to it.

Everything what you wrote is already there. Warning, off by default, created only if you create an automation, etc.

https://www.home-assistant.io/docs/automation/trigger/#webhook-trigger

And I saw this "only local" in the changelog from 2022.2.0 as well. At least I think so.

[esev]

The difference in this PR is that GET requests have no CORS preflight checks. The GET verb was intended to be a read-only action without side-effects. By adding GET, the webhooks are automatically triggerable from any static web page. That was not possible before this PR; you had to use some javascript or user interaction to trigger the webhook from a web page.

Example: With this PR, if I add a webhook with the ID Open_Garage. All a bad actor needs to do is trigger a GET to http://homeassistant.local:8123/api/webhook/Open_Garage. This can be in an email, a malicious ad, a URL shortener service, a PDF/document, a wiki/forum post that allows images, or any website. The user only needs to open the web page from a device on their local network, and doesn't need to interact with the web page. And no javascript is required. This bypasses the browser's built-in security mechanisms. And there was no indication in the UI that this might be unsafe.

This can't be prevented by checking if the device is on the local subnet. Until https://wicg.github.io/private-network-access/ is implemented in browsers, all that is required is that your browser (likely on the same subnet as HA) be able to access Home Assistant locally. And that only considers browsers. It could still be triggered by malicious content in document/PDF viewers, native apps, chat messages, etc.

Admittedly though, I think most browsers would block this behavior from secure sites. Browsers tend to not allow mixed content; http requests from https sites. So that would mitigate the risk a bit. But it would be great if we could also consider users who setup SSL for themselves too, and make them safe also.

Personally I'd like to see:

• The cors_allowed = True be an option on each webhook individually, and for the default to be False (not related to this PR, but it would close the pre-existing CSRF browser-based security issue by default)
• Another per-webhook option (Allow Web Browsers) to allow common browser User Agents to be permitted to trigger the webhook. And for that option to default to False. Or, as @davet2001 suggested, only allow specific User Agents. (Could be used to make this PR safer, but not a bullet-proof solution)
• For this PR specifically: A third per-webhook option to allow 'GET' requests to trigger a webhook, as these bypass the CORS preflight checks in the browser. Again, I'd like to see this default to False.
• Add an option to restrict the webhook to a specific IP address/ranges (suggested by @davet2001). Maybe default to the local subnet, but with the understanding that it doesn't provide protection for browsers in the default state (could be used to make this PR safer as long as the device at that IP isn't running a browser. Would be a big win for users who expose HA directly to the internet. Probably needs to consider reverse proxies: 'X-Forwarded-For')
• Provide a default, randomly-generated, webhook ID in the UI. Add a warning in the UI if the webhook ID doesn't contain much entropy/randomness. Similar to how some websites have password complexity tests. (not directly related to this PR)
• Disallow a webhook ID to be defined in a blueprint. Randomly generate one or let the user choose one. (assumes blueprints can define webhook triggers - I haven't verified this. This isn't directly related to this PR)

Each of these options could be described in the docs with examples of how they could be abused if enabled.

I'm not on the HA core development team, so don't take anything I've written as a requirement for this PR. I'm not suggesting the PR be blocked; I see how it is very useful. I'm only suggesting ideas to make things safe by default (from a browser security perspective) when using webhooks so users can enable them without adding security risks. IMHO I don't think users understand/consider the CSRF implications for browsers before using webhooks. I'd just like users to be aware of the danger, and need to perform an extra action, before using webhooks in an unsafe way. In it's current form, this PR increases the danger by bypassing CORS preflight checks in the browser, without providing any safe-by-default options. This increases risk for all previously defined webhooks that only expected POST/PUT/HEAD.

[alderete]

As someone who has requested GET support for webhooks, in several places, I think the additions proposed by @esev all sound like good ideas. In particular, the last four bullets:

• Per-webhook option for GET — false by default
• Per-webhook option for array of allowed IP addresses — local subnet by default?
• In the Automation editor UI, generate a random webhook ID by default (but allow me to override)
• Disallow webhook ID in blueprints, replace with random ID

All sound like terrific, and hopefully not terribly complicated additional development.

I'm sad that that might mean this PR requires some re-work. I have Shelly Button 1's that want GET webhooks today to make it easier for my wife to turn off the "smart" bedroom light switch without getting out of bed. (The switch is by the door; the bed is not.)

But safety first!

I'll personally commit to writing the doc to go with the changes, if the appropriate person wants to loop me in. (I'm a tech writer by day, and I have one commit for the webhooks doc already.)

[esev]

@alderete I've sent a PR for randomizing the webhook_id by default. https://github.com/home-assistant/frontend/pull/11568 Could you take a look, and update the docs if you think there is anything needed for that?

The copy button on the right-side copies the URL for the webhook to the clipboard (http://homeassistant.local:8123/api/webhook/bnWvYB5KLRBbkT9XMCl9zXoX). I figured that makes it a bit easier to use too.

Thanks for volunteering to update the docs!

[alderete]

I just submitted a PR for the webhooks doc: https://github.com/home-assistant/home-assistant.io/pull/21506

I attempted to add security details in the form of best practices and considerations (don't open your garage door, etc.), and other concerns discussed in this thread.

And I'm happy to continue to update the docs as this and other open webhook-related PRs move forward. (That is, please don't let doc be the blocker on this PR. I will fix it if that's what's needed.)

[esev]

I'm putting together a PR to:

1. Add an option to allow webhook triggers to be accessed from the internet, and making the default False.
2. Make all HTTP methods optional, and enabling enabling 'POST' & 'PUT' by default.

For 1, I'm making it so that Nabu Casa's webhooks still work regardless & any webhooks triggered via the websocket API still work too. For both of these, the user would have needed to explicitly enable it, so that seems safe to me.

Number 2 will be easy to adapt for this PR, and add 'GET' as an option too.

trigger:
  - platform: webhook
    webhook_id: bnWvYB5KLRBbkT9XMCl9zXoX
    allow_methods:
      - PUT
      - POST
    allow_internet: false

I'm going to wait to see if the core developers give any feedback on the two outstanding PRs, or this PR, before proceeding on this next change though. I'll need some help updating the docs and creating a note about this being a breaking change too, assuming this all looks okay.

[balloob]

I like the idea of opt-in GET. We should make sure it applies to the hooks coming in via cloud/websocket too.

[esev]

I added three PRs to move this forward. Please take a look and give feedback on the individual PRs.

1. Core: https://github.com/home-assistant/core/pull/66494
2. Frontend: https://github.com/home-assistant/frontend/pull/11680
3. Docs: https://github.com/home-assistant/home-assistant.io/pull/21637

automation:
  trigger:
    - platform: webhook
      webhook_id: "some_hook_id"
      allowed_methods:
        - POST
        - PUT
      local_only: true

[nadimaj]

Great ideas here. All being ignored though.

[davet2001]

Great ideas here. All being ignored though.

There are currently 370 open PRs... maybe you could help by reviewing the PRs that @esev has kindly created? If not the code PRs, maybe the documentation? Everything helps!

[FutureTense]

Is anyone working on this? It looks like a simple issue if you use a long enough random string as the header security shouldn’t be a problem

header = ''.join(random.choices(string.ascii_uppercase + string.digits, k=15))

[byroncoetsee]

Are we saying that it's better to strip ability and responsibility rather than give someone an opt-in feature (which could have like 3 warning popups if you're that concerned about this stranger's inability to make their own decisions)?

[davet2001]

@byroncoetsee @FutureTense I don't think anyone is objecting to this feature.

It's just not been reviewed yet.

You could help by reviewing the PRs - even a partial review helps. 😀

[byroncoetsee]

@davet2001 reviews added and awaiting merge... 👍🏼

[esev]

Are we saying that it's better to strip ability and responsibility rather than give someone an opt-in feature (which could have like 3 warning popups if you're that concerned about this stranger's inability to make their own decisions)?

I'd like to have this as an opt-in feature so that folks who previously setup webhooks are not impacted by this change. This PR will decrease security for all previously added webhooks. That's why I proposed the other PRs, to give users an option to enable GET if they need it.

[CR6-SE]

Any Update in merge?

[byroncoetsee]

@davet2001 juuuust need to hit the merge button 🥹

[CR6-SE]

@davet2001 juuuust need to hit the merge button 🥹

then please do it😍 A whole new world will be open for Doorbird🙃

[Anks329]

Is there any update to this PR? Would love to see this functionality merged and made available.

[Holland1]

Similar situation. Mobotix doorstation, only supports GET.

So upvote for this feature