Unavailable or unknown calculated values should adhere to the home assistant auto discovery specification

[FSeidinger]

Describe the bug

The ecowitt2mqtt app makes use of calculated values. Some of these values might be, depending on their nature, not available or not applicable. The Simmerindex is an example fo such a calculated value.

Currently ecowitt2mqtt exports not applicable, calculated values in HASS discovery mode through the state topic with the value of unknown. This leads to a problem, if the state is announced as a numeric value, probably having a unit of measure. The value unknown simply cannot be converted to a number.

If I interpret the integrations specification of home assistant sensors right, a (temporarily) unavailable sensor should be announced by the availability topic either as default online or offline or as a template for a available/non available payload through payload_available and payload_not_available as described in the availability section.

Why this might be important? I use the home assistant auto discovery with third party products which may have a stricter interpretation and simply give log errors, that unknown cannot be converted/imported to a number.

Steps to reproduce

1. Install ecowitt2mqtt with calculated values enabled.
2. Wait for a payload to arrive.
3. Watch out for the unknown state:

   simmerindex:
       config: {
           "availability_topic": "homeassistant/sensor/80A4CECA680304DDA6BAD8DD375C505B/simmerindex/availability",
           "device": {
               "identifiers": [
                   "80A4CECA680304DDA6BAD8DD375C505B"
               ],
               "manufacturer": "Ecowitt",
               "model": "GW2000A",
               "name": "GW2000A",
               "sw_version": "GW2000A_V2.1.9"
           },
           "json_attributes_topic": "homeassistant/sensor/80A4CECA680304DDA6BAD8DD375C505B/simmerindex/attributes",
           "name": "simmerindex",
           "qos": 1,
           "retain": false,
           "state_topic": "homeassistant/sensor/80A4CECA680304DDA6BAD8DD375C505B/simmerindex/state",
           "unique_id": "80A4CECA680304DDA6BAD8DD375C505B_simmerindex",
           "unit_of_measurement": "°C",
           "device_class": "temperature",
           "state_class": "measurement"
       }
   availability: online
   attributes: {}
   state: unknown
   

Possible solution

Just set the availability topic to offline. It might be also necessary to omit the state topic then.

simmerindex:
    config: {
        "availability_topic": "homeassistant/sensor/80A4CECA680304DDA6BAD8DD375C505B/simmerindex/availability",
        "device": {
            "identifiers": [
                "80A4CECA680304DDA6BAD8DD375C505B"
            ],
            "manufacturer": "Ecowitt",
            "model": "GW2000A",
            "name": "GW2000A",
            "sw_version": "GW2000A_V2.1.9"
        },
        "json_attributes_topic": "homeassistant/sensor/80A4CECA680304DDA6BAD8DD375C505B/simmerindex/attributes",
        "name": "simmerindex",
        "qos": 1,
        "retain": false,
        "state_topic": "homeassistant/sensor/80A4CECA680304DDA6BAD8DD375C505B/simmerindex/state",
        "unique_id": "80A4CECA680304DDA6BAD8DD375C505B_simmerindex",
        "unit_of_measurement": "°C",
        "device_class": "temperature",
        "state_class": "measurement"
    }
availability: offline
attributes: {}

Diagnostics

Not applicable.

Additional context

Not applicable.

[bachya]

We used to do this (mark sensors as unavailable if their underlying data didn't exist), but we switched to an unknown state instead because of https://github.com/bachya/ecowitt2mqtt/issues/293 (CC: @Anto79-ops).

I suppose we could do both (unknown state and mark the entity as unavailable), but first:

I use the home assistant auto discovery with third party products which may have a stricter interpretation and simply give log errors, that unknown cannot be converted/imported to a number.

I'm somewhat hesitant to make changes to the Home Assistant MQTT Discovery standard used here for some other product that isn't designed to ingest it. unknown is a valid Home Assistant state, so it's a reasonable state value in MQTT.

Can you share more details?

[Anto79-ops]

Yeah I agreed anything dealing with HASS Discovery should follow the standards there however Aaron is known for his magic and he could probably accommodate anything and make everybody happy :)

[FSeidinger]

We used to do this (mark sensors as unavailable if their underlying data didn't exist), but we switched to an unknown state instead because of #293 (CC: @Anto79-ops).

Didn't see that. And I guess it is due to the fact, that home assistant component can use basically three methods of indicating (temporarily) unavailable states. I've read the issue and the specs and it is still not clear to me, what a none playload is. I simply thought it is the payload topic with empty content.

I suppose we could do both (unknown state and mark the entity as unavailable), but first:

I use the home assistant auto discovery with third party products which may have a stricter interpretation and simply give log errors, that unknown cannot be converted/imported to a number.

I'm somewhat hesitant to make changes to the Home Assistant MQTT Discovery standard used here for some other product that isn't designed to ingest it. unknown is a valid Home Assistant state, so it's a reasonable state value in MQTT.

Can you share more details?

Sure. I use a WS90 weather station connected to the GW2000A base station. Using your wonderful mqtt exporter, I can connect the gateway to MQTT and from there pull it into my openHAB smarthome platform.

But lets just say that the quality of the openHAB MQTT binding is currently not high enough to cope with unavailable values. To get my own picture, I've read the home assistant manual and found out that there are basically three methods for home assistant components to signal unavailable states.

After trying out all three methods, I came to the conclusion, that none of the methods is currently supported by the openHAB binding. And therefore any configuration on your side resulting in a different rendering of the payload, will not work in my use case.

So I filed a bug on the openHAB MQTT binding and hopefully will get a corrected software in 3-12 months. Up to that point, I just have to live with the inconvenience.

And I'm also willing to close this issue, If you have no objection. But I will wait on your comment first.

[bachya]

Thanks for the info, @FSeidinger. A couple of thoughts:

1. If you specify an MQTT topic and don't enable Home Assistant MQTT Discovery, you'll get a payload that looks like this:

{
    "runtime": 0,
    "tempin": 90.0,
    "humidityin": 67,
    "baromrel": 29.769,
    "baromabs": 29.769,
    "temp": 90.0,
    "humidity": 73,
    "winddir": 287,
    "windspeed": 0.0,
    "windgust": 0.0,
    "maxdailygust": 2.24,
    "solarradiation": 88.67,
    "uv": 0,
    "rainrate": 0.0,
    "eventrain": 0.0,
    "hourlyrain": 0.0,
    "dailyrain": 0.0,
    "weeklyrain": 0.0,
    "monthlyrain": 2.012,
    "yearlyrain": 2.343,
    "soilmoisture1": 45,
    "pm25_ch1": 102.0,
    "pm25_avg_24h_ch1": 117.7,
    "wh65batt": "OFF",
    "wh25batt": "OFF",
    "soilbatt1": 1.5,
    "pm25batt1": 80,
    "beaufortscale": 0,
    "dewpoint": 80.2,
    "feelslike": 108.0,
    "frostpoint": 71.8,
    "frostrisk": "No risk",
    "heatindex": 108.0,
    "humidityabs": 0.0,
    "humidityabsin": 0.0,
    "safe_exposure_time_skin_type_1": null,
    "safe_exposure_time_skin_type_2": null,
    "safe_exposure_time_skin_type_3": null,
    "safe_exposure_time_skin_type_4": null,
    "safe_exposure_time_skin_type_5": null,
    "safe_exposure_time_skin_type_6": null,
    "simmerindex": 112.0,
    "simmerzone": "Caution: Heat exhaustion",
    "solarradiation_lux": 11224.1,
    "solarradiation_perceived": 81.0,
    "thermalperception": "Severely high",
    "windchill": null
}

Would ingesting this directly into openHAB (vs. trying to shoehorn the Home Assistant topics and their payloads) help? You don't get a lot of surrounding data (units, etc.), but the raw data is there.

2. If the above isn't enough, would it make sense to develop a separate "publisher" for openHAB (e.g., --openhab)? Put another way, is there an openHAB standard (like Home Assistant MQTT Discovery) that we should look to support?

[bachya]

Closing this as stale due to inactivity (somehow, my stale bot missed it). Feel free to comment if you would like to continue the conversation.