convention
convention copied to clipboard
homieiot
Retained set commands
Is there a reason, why set command are not allowed to be retained?
When I implemented my first device using mqtt, I liked the possibility to retain the color value send from openhab in mqtt, because my device was able to regain the value after it lost power or connection.
The only homie compliant way to achieve this is using a rule in openhab to detect the device coming back online. Or maybe subscribe to the retained property (without /set), to get the last value.
That indeed seems like a not yet covered use case for Homie. Thanks for bringing up this logical issue.
So far the set command is intended as a one-time event to instruct the device. A set command does not (necessarily) define the actual state of the device at a given point, nor should it be valid and wrongfully (?) be re-evaluated at a later point. Because of that reasoning the set topic is non-retained.
The only valid use of a re-evaluated (retained?) set command is the case of "Restore state after disconnect/power-off" as you described it. I don't like the idea to abuse the set topic by sending retained messages though.
We should discuss alternative solutions.
Remember that retained messages are only sent by the broker in response to a new subscription. So if Homie doesn't reconnect with a clean session, then the retained messages won't be resent. If Homie does connect with a clean session then any QoS 1 and 2 messages waiting for it will be discarded.
The property values are retained (or allowed to be retained). A homie device could after initialization restore its internal states by those retained values (if it wants too). Is that not enough?
@davidgraeff I believe everyone agrees there. If I am correct, then @MofX's intend was to discuss the use case and define a solution, e.g. in the implementation guideline for clients or in the FAQ. Correct?
@davidgraeff How can I retain the property values?
homie/devicename/nodename/property/$retained -->true
Is this the way to do it? It isn't working for me.
@riemp If it works depends on your implementation and not on the Homie convention. You are in the wrong repository.
The convention states that if $retained is set to true, then the implementation is required to retain the mqtt topic value.
I'm not taking about the controller. The controller is using the /set topic. I'm talking about your client IoT device esp8266 or whatever.
As you can see in this topic, we agreed that /set topics will never be retained.
@davidgraeff I have 3 light devices in my esp32(as 3 nodes --channels ) .and I'm using openhab to control them ,I'm able to make esp discoverable to openhab (as thing /by following homie convention) and also able to control it from openhab . esp is subscribed the topic homie/devicename/nodename/property/set
it works fine but .but I need my esp32 to go-to the last knwn state while it boots/whenever it reconnect to the broker
So I thought retainig /set topics from openhab is the best way to implement this.
Can u please suggest me any other way to do the same ?!
Disclaimer: I'm a bit in a bad mood because its raining here in my hometown.
But please RESPECT other peoples time. I could make beautiful open source commitments right now, instead I'm responding to a question of yours that is answered just one page of scrolling up. I quote myself:
The property values are retained (or allowed to be retained). A homie device could after initialization restore its internal states by those retained values (if it wants too). Is that not enough?
I like the idea that one can have Setpoint-Values and Measured-values. Both retained. This would cover an aspect, also mentioned here: https://github.com/kollokollo/MQTT-Hyperdash/blob/master/doc/MQTT-dashgen-naming-conventions.md
Maybe one can bring this together. I do not care if the suffix for a setpoint value is "_AC" or "/set". Also I like the idea to treat topic names as measured values by default (suffix not needed).
I wouldn't make this part of the core specification because I can think of use-cases where you do not want /set topics to be retained.
Since we do have the extension system in Homie now, I would suggest an "org.homie.retained-set" extension. It should talk about topic priorities (if same /set and / topics differ) and about implications for controllers and devices.
Extensions good and fine, we should not misuse the set topic in this way. Reason given above: https://github.com/homieiot/convention/issues/150#issuecomment-449852186
Imho we should clarify this point (it would logically be wrong) in documentation.
A retained set topic would play the role of an unconditional default value, rather than generating a new value or restoring the last state of a property from the broker topic (depending on the client implementation). If an unconditional default value is desired as a functionality of a Homie device property this should simply be implemented as an additional property. No change to the convention needed.
Probably there is no change in the convention needed because you can do it anyways with a different topic. It is just a way to make automation easy: The general imperative to any device would be: Try to make the measurement value equal or come as close as possible to the corresponding setpoint value. And therefor the setpoint value must be present (retained) all the time. This could be fundamental to IoT. But maybe its just one way to look at it.
Abuse of retained messages is one of my pet hates in MQTT. Their purpose is only for "what should should be sent to device when they create a new subscription to this topic", i.e. when a new consumer wants access to the telemetry. Existing devices restoring their connection will not be sent retained messages that they have previously received. They will only receive messages that were sent while they were offline if the messages are QoS 1 or 2. Existing devices restoring their connection should not be using clean session otherwise those QoS 1 or 2 messages will be lost. Equally they should not be subscribing to their own telemetry topics.
If you want to use MQTT to store the device internal state, then I would create a separate topic that contains the entire device state retained as a JSON or binary message, then subscribe to it only when you want to restore the state. Otherwise, you risk getting inconsistent/incomplete state information from the broker from the command or telemetry topics. This is an implementation detail for the device firmware and nothing to do with the OpenHAB (or equivalent) controller.
Well, there is no (generic) way to poll the last seen value of a topic from the broker other than using retained messages. You could implement an explicit get operation on the device, but this violates the decoupling of consumer from producer. Assume 1000 consumers connect to the broker, they initially do not know about the topics content, so they have to ask the producer device to resent it, individually. This way the load on the producer depends on the number of consumers (like on server-client models). One of the strengths on MQTT is that this is not necessary. Storage abuse is normally handeled by the broker using an overal or per-user quota. That looks OK to me.
Well, there is no (generic) way to poll the last seen value of a topic from the broker other than using retained messages.
It's fine to use retained messages to provide the last value to new consumers, where it makes sense. But MQTT is designed to be a lightweight message queue not a database.
Also, the set command has only one consumer---the target device---so once it's been processed, there's no need to keep it.
Finally, if there's is information that a device needs storing in order to work correctly, then that device should take responsibility for ensuring that it's stored, and not rely on third parties storing it correctly for them. For example, there's no guarantee that the command message is even valid, let alone retained.
Thankas @jamesmyatt. So in summary, how do we think should we document this topic, or does it need any documentation at all? Imho every topic coming up as an issue must be addressed in documentation, otherwise it will come up again and again.
Also, the set command has only one consumer---the target device---so once it's been processed, there's no need to keep it.
But what if the device itself gets offline and then reconnected? It needs to find out what the setpoint should be. How can he do this, not knowing who did the last /set command? If it has sored the last seen value by itself on the device FLASH memoy, OK, but what if the /set was changed during the time it was not connected? (assuming qos0)
Also, the set command has only one consumer---the target device---so once it's been processed, there's no need to keep it.
But what if the device itself gets offline and then reconnected? It needs to find out what the setpoint should be. How can he do this, not knowing who did the last /set command? If it has sored the last seen value by itself on the device FLASH memoy, OK, but what if the /set was changed during the time it was not connected? (assuming qos0)
If the message was sent while the device was offline, then it should use QoS 1 or 2. If it's QoS 0, then you can assume it wasn't important and you're still meeting requirements by dropping the message. You also may not receive a retained QoS 0 message when reconnecting unless you remove the subscription (unsubscribe or "clean session") and resubscribe to the topic.
If the message was sent while the device was online, then you ought to store the state yourself. FLASH/EEPROM would be the most reliable way since you're in total control of that. But if you can't do that (no non-volatile storage or you want to limit the number of writes), then I'd suggest a separate internal state topic as discussed above (https://github.com/homieiot/convention/issues/150#issuecomment-581830403); although, there's no guarantee that a third-party won't interfere with it.
It's also worth remembering that plenty of devices have multiple interfaces, e.g. MQTT + physical. Hence, if you use a retained /set message, then a physical button (or timer or HTTP API or whatever) to change the state to something else, then the MQTT set message now wrong. Equally, the contents of the /set message might not directly reflect the actual status, e.g. your command could be "toggle" (although that's probably bad practice anyway since it's not idempotent, but you get the idea) and it would make no sense for restoring.
Ok, I agree. But just for the fun of discussion: If the device also had a hardware button to change the setpoint, why not also publish itself to /set. This way it would be consistent, and in addition reflect the state of the hardware button. I know, there are many ways to do it. Maybe looking at real-life examples?
@kollokollo you are circling. Let's establish that the set topic is only written to to command the device (no retained state purpose) - why would the device itself bother to do so?
Once again: If you are interested in a default value or for your purpose a retained setpoint, define a dedicated attribute for it. That is by far the cleanest way to stay within the logical framework
@ThomDietrich , there seems to be a general lack of MQTT best practice guides, as far as I can tell. Might be worth having separately from the Homie site.
Excellent! I was just about to propose that we should start documenting these "Why does the convention do thing X like that and how am I supposed to solve use case Y around that?". This should imho be a document next to the convention:
MOVED to #51
We already started to discuss this over in https://github.com/homieiot/convention/issues/51
For my use case, set commands cannot be retained. My lighting is connected to OpenHAB via Homie. If the connection between OH and my lighting controller is down and lights in the home are controlled manually, and then the connection is restored, any set commands are resent and the lights are changed to whatever OpenHAB last set to them. Not desirable.
@mjcumming Not sure but if the physical state of your home and the virtual state are not in sync that sounds like a design error to me. Your light switches should update OpenHAB (I know that might be hard to do).
@Thalhammer The above example is sub-optimal because of this, yes. But there are a lot of other similar cases that are still problematic even if the state is always in sync.
Say I have a music player, and I want to play a song. But the music player fails to do so for some reason and thus updates the current playing item to none. If /set-messages are retained, the state could look like this:
-
homie/pi/player/song → none -
home/pi/player/song/set → foo.mp3
If now the music player reboots or reconnects for some other reason, it will receive the song command again and try to play that file. If it succeeds this time, it will suddenly and unexpectedly start playing music, possibly hours later.
@Thalhammer the issue occurs when there a loss of connection between the device and the controller (OpenHAB). If that connection is lost and then restored, any set messages from OH will be delivered and this behavior (for my use case) does not work as the state of the device may have changed and the set command from OH is no longer valid.
We need to solve for essentially (I think) 2 use cases when there a disruption in communication between a controller and a device....
- The controller is the source of truth and the device should always set it self using last set command from the controller
- The device is the source of truth and any set command from a controller is only valid for a short interval
Does that cover all the relevant scenarios?