Implement Homie for MQTT

[rkoshak]

Implement a Homie communicator to enable automatic discovery of sensor_reporter instances.

[evotodi]

I will second this as it would allow sensor discovery in Openhab. May I suggest https://pypi.org/project/homie-spec/ for the implementation. I have found it rather easy to use.

[rkoshak]

It's still on the list of things to do but not high on the list at the moment. I just went through a rewrite of sensor_reporter and I fear it will require another rewrite to support homie because it will need to know about all the sensors and actuators up front. And I'll have to add additional information to each add-on so it can describe the type and other information needed by Homie. The current architecture doesn't support that.

Thanks for the library though, I'll definitely look into that. This will get done, just not soon.

[DanielDecker]

I think the discussion about sensor / actuator discover API fits better in this topic, since it is close related to homie.

So sensors and actuators would get an interface to query their property e. g. datatype, measurement unit.

Or maybe it needs to be a combination of the two where the config on the actuator gets passed through to the Homie connection and that becomes our API.

I agree with that. Some homie properties are fixed for a sensor / actuator and could be hard coded. But it will be hard to combine the configured parameter and the hard coded ones. ~~I was thinking of a new class which holds all the possible properties that are necessary e. g. for homie. While the properties should be general enough so other systems with device discovery can make use of it. The idea is each sensor / actuator crates a child of that class and fills in known parameters. Later a connector e. h. "homie conn" can loop thru all devices and query the properties.~~

A good first solution might be to store the necessary homie properties in the configuration:

Connections:
    Homie:
        topic: main-light
        name: Living-room main light
        type: light
        cmd_src:
            topic: cmd
            name: set main light
            datatype: boolean
            settable: true
        destination:
            topic: state
            name: current light state
            datatype: boolean

So the device (e. g. a actuator) could have a function to return the necessary part of the configured connections. E. g. the homie connector would query all devices for the configured 'homie connections'. The datatype and the property 'settable' will be the same for all actuators. Maybe in a later step we can make them a default, so they can be removed from the configuration.

[rkoshak]

A lot of that information is going to be hard coded in the actuator. For example, a GPIO actuator could only ever be a boolean and the very fact that it is an actuator means that it is settable.

I'm thinking something like the following.

There is already a step for actuators where they register with the connections. This is how they tell the connection which MQTT Topic/openHAB Item to monitor for commands. We could add a similar registration to sensors (which would be a noop for MQTT, openHAB and local connections) and have the actuators and sensors pass the whole config for that actuator on registration. Then the Homie connection can publish what it needs to to the relevant topics at that time for discovery.

But not everything needs to be defined in that yaml config file. Each sensor and actuator knows a bit about itself. For example a GPIO actuator can only really be some variation of a boolean or a toggle. So we don't need to have the user tell the Homie connection that through the YAML. The sensor and actuator can add that part itself to what it passes to the connection when it registers.

Similarly I think Homie defines what the cmd and state topics need to be so we really only need the main topic and the rest can be assumed. Even if Homie doesn't specify that, we can take a stand here and create a standard that we follow.

So the config might be something like:

Actuator:
    # blah blah blah
    Connections:
        Homie:
            topic: main-light
            name: Living-room main light
            type: light

We know it's boolean because it's a GPIO actuator. We know the cmd and src stuff because we follow a standard for those. We know it's settable because it makes no sense to have an actuator that can't be commanded.

[DanielDecker]

There is already a step for actuators where they register with the connections. This is how they tell the connection which MQTT Topic/openHAB Item to monitor for commands. We could add a similar registration to sensors

Actually I thought the other way around: that the connection requests the sensor information via a public function of the sensor. But I think both approaches are equally good.

There is already a step for actuators where they register with the connections.

I assume you mean to have a new register function beside the existing register for actuators with the dedicated function to push the actuator properties to the connections.

We could add a similar registration to sensors (which would be a noop for MQTT, openHAB and local connections) and have the actuators and sensors pass the whole config for that actuator on registration.

I assume with config in this context you mean the sensor / actuator properties which partly include configured parameters from the YAML config. I was thinking of an object which will hold all possible properties for one sensor output, so a sensor with 3 outputs will have 3 of these objects. On register these objects are passed to the connectors.

But not everything needs to be defined in that yaml config file. Each sensor and actuator knows a bit about itself. For example a GPIO actuator can only really be some variation of a boolean or a toggle. So we don't need to have the user tell the Homie connection that through the YAML. The sensor and actuator can add that part itself to what it passes to the connection when it registers.

I agree. I firstly thought also this way and struggled then on the problem on how to combine YAML configured parameters with hard coded parameters. But your config example solves this in a easy way: It will not be possible for the user to name e. g. the CommandSrc topic since the name is hard coded.

[rkoshak]

Actually I thought the other way around: that the connection requests the sensor information via a public function of the sensor. But I think both approaches are equally good.

I agree both are equally good but the plumbing for having the devices push the config to the connections is already partially built so I think it'd be easier.

I assume you mean to have a new register function beside the existing register for actuators with the dedicated function to push the actuator properties to the connections.

No, I'm thinking replacing that function because the current one would be irrelevant. Right now the actuator calls register on the connection passing what we currently call the cmd_src. This tells the connection "when you receive a message on this Item/MQTT Topic, pass that message to me."

My proposal is instead of just passing the cmd_src we pass the whole YAML config plus any hard coded defaults to the connection. Then the connection can do what ever it needs to to configure the listener (and publisher?) based on that config.

We'd have to add a similar registration on the sensors too to support Homie I think.

I was thinking of an object which will hold all possible properties for one sensor output

I'm cautious here because I wouldn't want to hard code too much connection specific config stuff into the device classes. Right now the configs are almost just a pass through which is pretty powerful because we can add a new connection and not have to touch any other code. It's just a different config. I'd hate to lose that by adding special knowledge to the device classes that is really only relevant to the connections.

I can't tell if that's what you mean or not by "hold all possible properties".

While this approach is loosy-goosey and I wouldn't propose such in a lot of environments, I think flexibility here is more important than "good coding practices."

It will not be possible for the user to name e. g. the CommandSrc topic since the name is hard coded.

Correct but I think it's a reasonable trade in some cases. The approach can be made on a class by class basis I think. For example, the topic names for cmd and state could be hard coded under the configured root topic for MQTT for a GPIO switch but maybe the Govee sensor needs us to define all the reported sensors individually so it more easily supports local and openHAB connections.

[DanielDecker]

No, I'm thinking replacing that function because the current one would be irrelevant. Right now the actuator calls register on the connection passing what we currently call the cmd_src. This tells the connection "when you receive a message on this Item/MQTT Topic, pass that message to me."

My proposal is instead of just passing the cmd_src we pass the whole YAML config plus any hard coded defaults to the connection. Then the connection can do what ever it needs to to configure the listener (and publisher?) based on that config.

I see. Currently at #97 the register function passes the connection relevant dictionary to the connection. So with this configuration:

#definition of logging and connectors

SensorMainLightSwitch:
   Class: gpio.rpi_gpio.RpiGpioActuator
   Level: INFO
   Pin: 8
   Connections:
      local:
         Switch:
            StateDest: eg_k_greenLED
         LongButtonPress:
            StateDest: ge_k_long_press
      mqtt:
         ShortButtonPress:
            StateDest: main_switch/short_press
            Retain: yes
      openHAB:
         ShortButtonPress:
            Item: eg_k_short_press
      Homie:
         topic: main-light
         name: Living-room main light
         type: light

The homie connection would receive:

{  'topic': 'main-light',
    'name': 'Living-room main light'
    'type': 'light'  }

This is not the whole actuator config as you described but I think the homie connection wouldn't gain more from receiving the whole config. Is it enough if a connection receives only its sub tree form the connection section?

I'm cautious here because I wouldn't want to hard code too much connection specific config stuff into the device classes. Right now the configs are almost just a pass through which is pretty powerful because we can add a new connection and not have to touch any other code. It's just a different config. I'd hate to lose that by adding special knowledge to the device classes that is really only relevant to the connections.

I understand you want to keep core clear of connection related stuff. But I wounder where do you want to store the hard coded device information to pass it over to the connection on registration? That's where I first thought about a new object, which was created from a class which only purpose is to store some device related data. One solution to keep the flexibility would be that the device adds at init() the hard coded device properties to all subsections (connection name sections) of the 'Connections' section. So the homie connection would see this:

Homie:
        topic: main-light
        name: Living-room main light
        type: light
        cmd_src:
            topic: cmd
            name: set main light
            datatype: boolean
            settable: true
        destination:
            topic: state
            name: current light state
            datatype: boolean

While the user has configured only:

Homie:
        topic: main-light
        name: Living-room main light
        type: light

I'd hate to lose that by adding special knowledge to the device classes that is really only relevant to the connections.

I'm not sure what you mean with special knowledge. The device classes have to know if its e. g. 'settable', while this is only relevant for some connectors like homie.

While this approach is loosy-goosey and I wouldn't propose such in a lot of environments, I think flexibility here is more important than "good coding practices."

I agree, flexibility is more important.

[rkoshak]

Is it enough if a connection receives only its sub tree form the connection section?

Probably, unless we want the class to add stuff to it that might be relevant like whether it's a switch and other similar stuff which might be useful to fill in the stuff that Homie needs. The GPIO and exec devices are the only ones where that stuff is a little squishy so we'd need to provide more details in the config. But for sensors like Govee, each sensor is well defined and understood. If we can we should force the user to have to configure, for example that this part represent a temperature in °C and that part is % humidity and such.

The device classes have to know if its e. g. 'settable', while this is only relevant for some connectors like homie.

That is a problem, I agree. We need to find the middle path between pushing too much configuration on the user in cases where the device should know what it is and making too many changes that will be Homie specific.

But we have to get that information to Homie somehow. Maybe forcing the user to have to configure it is the most flexible way to do it. Short of that we'd have to create a whole custom set of metadata that connections like Homie (in the future others?) would have to interpret and translate.

We have the same problem even if we have an API that the connection calls on the devices instead of having this info pushed to them too.

[DanielDecker]

Probably, unless we want the class to add stuff to it that might be relevant ...

What class do you mean?

That is a problem, I agree. We need to find the middle path between pushing too much configuration on the user in cases where the device should know what it is and making too many changes that will be Homie specific.

I agree. We could find common device properties, that are required by e. g. homie and matter and hard code them in the devices. Something which is only homie specific would be configured manually in the config.

Maybe forcing the user to have to configure it is the most flexible way to do it.

Sure this will be very flexible and probably this could be the first step to get homie working. But I think if homie is way more complicated to configure than MQTT than many users will ignore the homie connector.

[rkoshak]

What class do you mean?

The device. The sensor or actuator. I'm thinking of cases like the Govee sensor which knows that, for example, one of it's channels is going to be a temperature in °C and another one is going to be % humidity. My though is why make the end user configure that in the YAML since it's already known by the sensor. The sensor could amend the dict parsed from the YAML with that information in some cases.