Add ESPhome discovery via MQTT

[Links2004]

Breaking change

Proposed change

This change will allow the ESPhome component to discover ESPhome devices that have the API enabled. This is usefull when mDNS is not possible in the Network, e.g. differnet subnet.

Type of change

• [ ] Dependency upgrade
• [ ] Bugfix (non-breaking change which fixes an issue)
• [ ] New integration (thank you!)
• [x] New feature (which adds functionality to an existing integration)
• [ ] Deprecation (breaking change to happen in the future)
• [ ] Breaking change (fix/feature causing existing functionality to break)
• [ ] Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• [x] The code change is tested and works locally.
• [x] Local tests pass. Your PR cannot be merged unless tests pass
• [x] There is no commented out code in this PR.
• [x] I have followed the development checklist
• [x] I have followed the perfect PR recommendations
• [x] The code has been formatted using Ruff (ruff format homeassistant tests)
• [x] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• [ ] Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• [x] The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• [ ] New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• [ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
• [ ] Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

• [ ] I have reviewed two other open pull requests in this repository.

[home-assistant[bot]]

Please take a look at the requested changes, and use the Ready for review button when you are done, thanks :+1:

Learn more about our pull request process.

[home-assistant[bot]]

Hey there @ottowinter, @jesserockz, @kbx81, @bdraco, mind taking a look at this pull request as it has been labeled with an integration (esphome) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of esphome can trigger bot actions by commenting:

• @home-assistant close Closes the pull request.
• @home-assistant rename Awesome new title Renames the pull request.
• @home-assistant reopen Reopen the pull request.
• @home-assistant unassign esphome Removes the current integration label and assignees on the pull request, add the integration domain after the command.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

[bdraco]

Please avoid doing force pushes once review has started as it means we have to start over and compare everything again

[Links2004]

ok noted, used to the rebase force push workflow from work.

[bdraco]

LGTM. I don't actually use mqtt with esphome so it will need another reviewer to take a look before merging

[Links2004]

will create a PR for ESPhome on the weekend to decouple MQTT entity discovery from IP / device discovery on the MQTT Component as an option, but the code here will work with out this change.

[emontnemery]

I'd like a second opinion on this before merging

[emontnemery]

@Links2004 I'm missing the full picture here. ESPHome has support for the Home Assistant MQTT protocol which does discovery on the homeassistant/# topic. In this PR, you add discovery of the native API on the esphome/discover/# topic.

Does this require changes also to ESPHome? You've opened PR https://github.com/esphome/esphome/pull/6673 which is related.

[bdraco]

https://github.com/esphome/esphome/blob/5edf4970bde1d2d5dda2fa34c1ba062a8f3aa8a1/esphome/components/mqtt/mqtt_client.cpp#L84

Relevant ESPHome code for context

[edenhaus]

This is usefull when mDNS is not possible in the Network, e.g. differnet subnet.

It's possible to have mdns working with different subnets/networks. You can use mdns reflectors for it.

Why would you use only mqtt for discovery and than switch to the native api instead of using mqtt for all the communication?

[emontnemery]

https://github.com/esphome/esphome/blob/5edf4970bde1d2d5dda2fa34c1ba062a8f3aa8a1/esphome/components/mqtt/mqtt_client.cpp#L84

Relevant ESPHome code for context

Thanks. That's doesn't seem to be documented, at least not here: https://esphome.io/components/mqtt.html Also, do we want discovery of the native API to not happen when discovery using Home Assistant MQTT protocol is enabled? Maybe that's not an issue since discovery over mDNS still happens?

[bdraco]

The ESPHome dashboard does support MQTT discovery for remote ESPHome devices that may be on different networks/over WAN

https://github.com/esphome/esphome/blob/5edf4970bde1d2d5dda2fa34c1ba062a8f3aa8a1/esphome/mqtt.py#L149 https://github.com/esphome/esphome/blob/dev/esphome/dashboard/status/mqtt.py

[bdraco]

Also, do we want discovery of the native API to not happen when discovery using Home Assistant MQTT protocol is enabled? Maybe that's not an issue since discovery over mDNS still happens?

I think it doesn't matter which method it comes in via if both are enabled since they both flow into async_step_discovery_confirm so the outcome should be the same.

[Links2004]

@emontnemery

there are 2 ways to connect ESPHome to Home Assistant.

1. via Home Assistant MQTT entity discovery e.g. https://www.home-assistant.io/integrations/switch.mqtt/ which is not touched in this PR

2. via ESPHome API which will allow Bluetooth Proxy, voice assistant this change will allow to discover the ESPHome devices via MQTT to add them via the ESPHome API

the changes in ESPHome make it easier to use, but its working with out the changes there.

@edenhaus

mDNS is no option in my setup (running home assistant in a kubernetes cluster) currently I always need to manually lookup IPs and copy them.

Side note: this was the reason I added the esphome/discover/# capabilities to ESPHome last year ;)

[emontnemery]

The ESPHome dashboard does support MQTT discovery for remote ESPHome devices that may be on different networks/over WAN

https://github.com/esphome/esphome/blob/5edf4970bde1d2d5dda2fa34c1ba062a8f3aa8a1/esphome/mqtt.py#L149 https://github.com/esphome/esphome/blob/dev/esphome/dashboard/status/mqtt.py

Thanks @bdraco 👍

[Links2004]

Have added the MQTT device discovery protocoll to the ESPHome docs: https://github.com/esphome/esphome-docs/pull/3809

https://deploy-preview-3809--esphome.netlify.app/components/mqtt#mqtt-device-discovery