Explicitly generate an _external_ webhook URL for Withings integration

[Flameeyes]

Proposed change

The current implementation of webhook.async_generate_url() will avoid using the cloud URL, generating an internal URL that Withings will not accept.

Force the URL to be generated disallowing internal URLs, and preferring cloud instead.

Note: this is more of a short-term fix, since this is causing issues for users of 2022.6. A more proper solution would be to have a selection of internal/external when generating the webhook.

Type of change

• [ ] Dependency upgrade
• [X] Bugfix (non-breaking change which fixes an issue)
• [ ] New integration (thank you!)
• [ ] New feature (which adds functionality to an existing integration)
• [ ] Breaking change (fix/feature causing existing functionality to break)
• [ ] Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #72614
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• [X] The code change is tested and works locally.
• [X] Local tests pass. Your PR cannot be merged unless tests pass
• [X] There is no commented out code in this PR.
• [X] I have followed the development checklist
• [X] The code has been formatted using Black (black --fast homeassistant tests)
• [ ] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• [ ] Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• [ ] The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• [ ] New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• [ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
• [ ] Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• [ ] No score or internal
• [ ] 🥈 Silver
• [ ] 🥇 Gold
• [ ] 🏆 Platinum

To help with the load of incoming pull requests:

• [ ] I have reviewed two other open pull requests in this repository.

[probot-home-assistant[bot]]

Hey there @vangorra, mind taking a look at this pull request as it has been labeled with an integration (withings) you are listed as a code owner for? Thanks! _{^{(message by CodeOwnersMention)}}

[Bluhme1]

Any news on this? The work-around works ok but it is not a real solution.

[nebriv]

@Flameeyes @vangorra

Alright I think I got this working. In addition the patch above I also had to effectively comment out this line: https://github.com/home-assistant/core/blob/dev/homeassistant/components/withings/init.py#L191

For some reason in the requests, maybe because its being proxied through nabu casa, the body_exists flag is not set.

These are the changes I made. Essentially just wrapping it in a try. I don't really know what the impact of doing this is though.

    try:
        # Handle http post calls to the path.
        if not request.body_exists:
            return json_message_response("No request body", message_code=12)
    except AttributeError:
        _LOGGER.error("Caught Attribute Error, missing request_body")
        _LOGGER.debug(dir(request))
        # Not really sure what this is really checking here.
        # return json_message_response("No request body", message_code=12)

This results in the following log output if it useful to anyone:

2022-06-15 20:43:21 ERROR (MainThread) [homeassistant.components.withings] Caught Attribute Error, missing request_body
2022-06-15 20:43:21 DEBUG (MainThread) [homeassistant.components.withings] ['__annotations__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_content', '_text', 'content', 'headers', 'json', 'method', 'mock_source', 'post', 'query', 'query_string', 'status', 'text', 'url']
2022-06-15 20:43:21 DEBUG (MainThread) [homeassistant.components.withings] Withings webhook triggered

[Flameeyes]

@nebriv thank you so much for testing this!

I've added a commit now that implements the body_exists attribute so that it matches the aiohttp class it's supposed to replace. Withings is the only webhook using it, so maybe removing the call (it does look superfluous to me) would be easier, but I'll leave that to @frenck to consider.

[nathan-curtis]

Do we have an ETA on this? The workarounds isn't working for me on 2022.7 and no combination of base URL seems to be getting it back now. We've been waiting on this since February.

If you need help testing let us know what to test. But guys were stuck out here..

[nebriv]

How long do these changes typically take to get approved and merged in? I'm kind of stuck not being able to update HomeAssistant until this patched. 😦

I know there are a lot of pending review items - and appreciate the immense amount of work it takes triaging, reviewing, and implementing everything... especially for the low low price of free... But I'd love to see this one resolved!

Thanks all!

[nebriv]

@frenck sorry for the ping - but how can we move this PR forward?

[oneseventhree]

This is still an issue. Withings Bed Sensor only works if I manually put in the External URL to my Nabu Casa.

[LCMilstein]

I am also still having trouble here. I have 2 sensors. 1 of them is installed and works (I have to lay in bed after a restart), but it works. The other, I removed (by accident) and I can NOT add the integration. It continues to report: {"errors":[{"message":"redirect_uri_mismatch: The redirect URI provided is missing or does not match partner callback url"}]} Nothing is showing in the logs. I've tried manually including the config in yaml (and then I get a warning that's been deprecated and I should remove it, but doesn't matter whether it's in or not). I've also tried manually modifying the const.py, manifest.json and common.py for the withings component... No change.

[smileyhogue]

This is still an issue. I tried the work around that suggested using the generated webhook for the callback URL and that also did nothing.

I am also getting the 293 error code.

[pabloromeo]

The only way I was able to get the integration to work, was by following the steps here: https://community.home-assistant.io/t/withings-integration-2022-4-2-redirect-uri-mismatch-error/410861/18

Basically manually re-enabled the "my:" integration in the config file, and then changing the callback URI in https://developer.withings.com/dashboard/ and setting the callback to https://my.home-assistant.io/redirect/oauth

[smileyhogue]

For anyone using Nabu Casa, after 3 days of trial and error I have finally gotten it working. You seem to absolutely NOT be able to use your own domain at the moment. I had to change the external_url in configuration.yaml to use my nabucasa domain and after restarting, that worked for me. Though it still does fail occasionally after a restart of home assistant still.

[marciogranzotto]

@frenck @Flameeyes just changing homeassistant/components/webhook/__init__.py:73 async_generate_url to allow_cloud=True won't fix this issue?

[frenck]

I've marked this PR, as changes are requested that need to be processed. Please un-draft it once it is ready for review again by clicking the "Ready for review" button.

Thanks! 👍

../Frenck

Learn more about our pull request process.

[GarethOates]

Any update on the status of this PR? It seems like a lot of people are waiting for it to be resolved.

[LCMilstein]

@frenck not sure if the webhook issue here is the only one, but confirming that Withings integration setup continues to fail for the in_bed binary sensor, reporting unavailable 100% of the time (not related to restarting HA, nor failing to initiate by lying in bed -- just fully will never come back with data to Home Assistant even though data is being logged into the Healthmate app).

[johntdyer]

@Flameeyes any change you can make the requested changes so we can get this one over the hump and into a release? Thanks !!

[joostlek]

As update, I hope to have Withings reworked in the next release, I already made major steps and this will be one of the features.

[oneseventhree]

As update, I hope to have Withings reworked in the next release, I already made major steps and this will be one of the features.

The birthday release or the one after :)?

[joostlek]

hehe there is no birthday release. I am aiming for 2023.10

[joostlek]

Will be fixed in #100916. Closing this one as it is too far behind at this point. Thanks for all the efforts ❤️