feature-requests
feature-requests copied to clipboard
esphome
Create Native API Server
Problem with current native API is that the ESP is the server. That was an intentional decision that makes setup much easier (and so the default will stay that way).
However, for things like deep sleep the api needs to be turned around: A central server (kind of like MQTT broker) that all ESPs connect to. Current plan is to incorporate this new server directly into HA.
With both client and server support, will you add support for direct calls between nodes?
Would it be possible to use the built-in API that home assistant already has? You could call it via a rest call and make state changes in it very easily. Authentication info would need to be provided through the config. Might be easier vs building your own server.
No, that is not real time and a hacky solution. I want to build a solution that will last, not one that will have to be replaced in the next release. Besides, most of the API code more or less exists already, the "server" we're speaking about here would not even have to understand what's inside the messages it is sending/receiving, it just needs to read them from ESP and relay them to subscribed clients like HA
I'm trying to create battery based solution like in #174. Any ideas to push this forward?
While I understand the main use case here is for deep sleep, another benefit would be faster time to HA frontend control for nodes that are powered on. For example, a light bulb that is powered off after a while and then powered back on takes some time to be picked up be HA (my understanding is that this is due to the exponential back off with the current API connection). For now I'll be moving to MQTT but I'm looking forward to this feature.
Another consideration is security. If ESP is not the server anymore, but the client, it has to initiate the connection to the server, which means the firewall has to allow the subnet, which is used by IOT devices to initiate new connections to the main lan. This loosens up security a bit.
I think the best solution would be to have a home assistant integration that listens on a port and all esphome nodes can connect to it. Any transport layer security problems should be part of #133 and should not prevent this development. As far as I know it should be possible to pre-establish the socket connection and the use the existing gRPC inside. So the existing protocol can still be used. Any proposals to this idea?
Raphael
What I forgot to say: There should be some kind of an authentication of the clients. Therefore we could either use the proposed TLS-PSK from #133 or as long as this is non-existent one could use a key that is stored in HASS and esphome and when opening the connection this key is checked. It doesn´t sound ideal but without TLS there wont be a goodsolution
@ski7777 Otto already has some ideas of how he's going to do it, however he is not doing it!
What makes this complicate for me is:
-
The current model should still work, discovery and connections initiated from ESPPlugin
-
ESPNode is a server, and thus more than one ESPPlugin can connect to it simultaneously and control it, this makes it hard to say to which server the ESPNode should connect when it's in client mode, it can connect to a fixed one, the last one or a list.
IMHO this api_server thing makes sense only for deep_sleep (or cold boots after a serial or OTA flash) so I'd go with a mechanism to ping ESPPlugin to connect to ESPNode, this should make the connection quicker, not as quick as running the comms in the connection initiated by the ESP, but much simple implementation, and if this fails for some reason all still works as usual.
A "server" on the ESPPlugin is also desired for OTA upgrades to sleeping nodes
@ski7777 Otto already has some ideas of how he's going to do it, however he is not doing it!
@glmnet What do you mean with he's not doing it? Did the idea get dropped?
My ideas are just a proposal. I think we should have a full concept first and then start thinking about the implementation. Everything else will cause frustration to the developers who's pull requests get rejected
@ski7777 Otto already has some ideas of how he's going to do it, however he is not doing it!
@glmnet What do you mean with he's not doing it? Did the idea get dropped?
He’s idle, fr is ok.
I would also be very happy to hear some news about this feature, as I think it is very crucial for low energy setups.
I think a simpler solution here would be a wakeup notification from the device to HA. Could be a simple http rest command that signal i'm awake, please connect to me. That way there is no major design changes required and we can quickly react to a woken up device.
I just relalize this is exactly what @glmnet proposed too. So what would be needed for this is to add a endpoint to the esphome component i HA that can trigger a connection to device.
Actually.. this ping function is likely already in place. The esp device will do a mdns broadcast on startup i think. So home assistant just need to react to that.
Well, I'm interested in "calling home" to allow devices to connect to HA from behind NAT (e.g. on cellular networks).
Actually.. this ping function is likely already in place. The esp device will do a mdns broadcast on startup i think. So home assistant just need to react to that.
@elupus yeah but that is all home assistant still list all valuses as unavalilible sometimes due to deep sleep. so i am afrait i just dont think this is fully implemented partialy at best :( or i am dummes person and i dont know tohandle deep sleep with native api not with mqtt.
Well, i think deep sleep behaviour should be improved for native api. When device goes to deep sleep, sensor entities aught not go unavailable until the deep sleep period ends plus some margin. They should just retain last reported value.
Then i think config entities should ve added, that remain available and get synced to the device once it boots.
Nine of above us currently supported. But would make it much easier to make use of deep sleep.
Actually.. this ping function is likely already in place. The esp device will do a mdns broadcast on startup i think. So home assistant just need to react to that.
This was implemented some time ago, however as noted by other people it's not very reliable as mdns is involved.
I stumbled over this after a few days wondering why my deep sleep sensors do not report their values within 15s on time. This should be documented as it makes deep sleep verry inefficient.
It would also mean that you could set a hostname for the server in the configuration and then the nodes can be anywhere, on the internet, via mobile, anything.
I was quite surprised to see that ESPHome relies on broadcasts, nodes with open ports and all nodes being on the same subnet. Not really appropriate anymore IMO.
However, the even better way is probably to implement MQTT over TLS and to teach the dashboard to do OTA over MQTT. That would instantly open up the world of MQTT bridges, clustering and a whole bunch of tooling.
@ski7777 Otto already has some ideas of how he's going to do it, however he is not doing it!
What makes this complicate for me is:
- The current model should still work, discovery and connections initiated from ESPPlugin
- ESPNode is a server, and thus more than one ESPPlugin can connect to it simultaneously and control it, this makes it hard to say to which server the ESPNode should connect when it's in client mode, it can connect to a fixed one, the last one or a list.
IMHO this api_server thing makes sense only for deep_sleep (or cold boots after a serial or OTA flash) so I'd go with a mechanism to ping ESPPlugin to connect to ESPNode, this should make the connection quicker, not as quick as running the comms in the connection initiated by the ESP, but much simple implementation, and if this fails for some reason all still works as usual.
A "server" on the ESPPlugin is also desired for OTA upgrades to sleeping nodes
I kind of agree here. I'm not really a fan of having both client and server modes unless it's really needed, as per the "Narrow waist" principle, it's really nice to have fewer possible configurations to avoid tools getting fragmented. Protocols with tons of optional modes usually wind up only using one of them anyway.
A simple reverse connection protocol would be pretty easy. Not sure if the APIs allow it, but you could store the last 2 or 3 client IP addresses in RTC ram, and unicast them an MDNS packet to make things more reliable.
Or just create a new UDP protocol that could carry the same data as the native API, after the crypto is already set up with the TCP version. There are already several components like the camera and microphone that could benefit from unreliable transport, so it could be used for more than just on/off notifications.
Any news/progress with this ?
My use case is: in some locations I would like to have an esphome device (ONLY). Home Assistant installation would be hosted in a cloud VM. The network, the esphome device sits in, only has Internet. In such case, esphome node should connect TO home assistant, not the other way around.
I'm aware "home assistant" means "using at home", with everything working offline, etc. But the system (hass + esphome especially) is so usable for so many things, I think implementing an option to provide esphome devices sitting somewhere where there's no homeassistant, it would be nice if they could connect to hass over the Internet. MQTT client in esphome can't do TLS (also can't do client certs), so that's not an option.
Native API client could also provide 99% of everything needed to enable node-to-node communication as well, which is another great thing, for projects with no need for a central hass.
I really see a native API client as a great upgrade, that would enable a whole new set of use cases and projects. I see it as an option, to add alongside existing native API server for projects that need it (and not to replace the current native API server, as others have mentioned, because that also has it's own positive use cases).
I am a developer but unfortunately I have near-zero knowledge of C... so can't help much with implementation. But I'd like to try and help if there's any way I could, perhaps with advice or design, I do have a bit of experience (20+ years) and I'm good in networking. If someone wants my help, let me know. I'd really love to see this feature move forward.
(edit for typpos)
