merbanan
Support for sensors with `state` keys
I have several sensors that show up fine on autodiscovery, but unfortunately the primary key used for the sensor is not in the list: https://github.com/pbkhrv/rtl_433-hass-addons/tree/main/rtl_433_mqtt_autodiscovery#what-sensorsdevices-does-this-script-support
My sensors look like this:
time = 2024-06-09 22:15:58
id = 309952
channel = 8
event = 128
state = open
contact_open = 1
reed_open = 0
alarm = 0
tamper = 0
battery_ok = 1
heartbeat = 0
mic = CRC
I'd like to request that the state key is added to the supported list. Other keys that would be helpful are contact_open, and reed_open, but those are generally redundant of state.
I'm not sure how to get these added, or what the best way to do this would be. I'm open to adding these manually as well, but I'm lacking a bit of understanding on how 'discovery' should work here. If I manually add mqtt sensors, do I not need to use autodiscovery (https://github.com/pbkhrv/rtl_433-hass-addons/tree/main/rtl_433_mqtt_autodiscovery#what-do-i-do-if-my-sensor-is-not-supported)?
Thanks for any help on this!
You have filed an issue in the rtl_433 repo. for that, you should be reading rtl_433_mqtt_relay and rtl4_433_mqtt_hass in examples. You may wish to modify them.
If a hass addon is not doing what you want, and you don't want to start reading python code here and using your own version, that should be addressed to that addons tracker.
I set the feedback tag to ask you to convert this into an issue that is about the contents of the rtl_433 repo. (if that's what you want to do vs closing and bringing it up with the addon).
That looks somewhat like the Honeywell/2GIG/etc. family of security sensors. You probably want to define a binary_sensor based off of contact_open.
My advice is not to use autodiscovery but to instead use manual MQTT configuration. There are two many things that autodiscovery doesn't know. Some sensors may use reed_open or alarm for the bit you are interested in, so the decoded state won't match. The autodiscovery script does have definitions for contact_open, reed_open, but they don't seem to get processed. I never tried to figure out why.
I like being able to set the device class to door, window, motion, etc. but have it defined as data someplace outside of home assistant's config_json files. There is more I could be doing to monitor availability like Zigbee2MQTT does.
I wound up defining all my sensors in a YAML file and then defined a Jinja2 template (same templating as home assistant) for the MQTT configuration. I process about 30 security sensors this way. If I need to update something I usually just need to change the template and run my script.
honeywell_sensors:
- esn: 123456
name: "Front Door"
type: "door"
model: "Honeywell 5800mini"
area: "Foyer"
description: "Main Entrance Active Door"
model_comment: "4th 5800mini"
zone: 1
- esn: 654321
name: "Guest Bedroom Window"
type: "Window"
model: "Versa Mini"
manufacturer: "Versa"
area: "Guest Bedroom"
description: "Guest Bedroom East Window"
model_comment: "Versa Mini, 1st from 2nd batch"
channel: 10
rtl_433_contact_field: "reed_open"
zone: 18
This is what the template file looks like. It's quick and dirty so it defined everything for 1 sensor in 1 file.
---
#
# Test Honeywell Security 5800 jinja template.
mqtt:
binary_sensor:
- name: "{{ contact_short_name | title }}" # TODO "Door"
object_id: "{{ device_name_safe + '_' + contact_short_name }}" # TODO "front_door_contact"
unique_id: "{{ device_unique_id + '_' + contact_short_name }}" # "rtl_433_honeywell_123456_contact"
state_topic: "{{ device_topic + '/' + contact_sub_topic }}" # "rtl_433/+/devices/Honeywell-Security/8/123456/contact_open"
payload_on: 1 # open
payload_off: 0 # closed
device_class: "{{ device_primary_class }}"
force_update: true # Needed?
expire_after: "{{ device_unavailable_seconds }}"
device:
name: "{{ device_name_pretty }}" # "Front Door Sensor"
model: "{{ device_model }}" # "5800 Mini"
manufacturer: "{{ device_manufacturer }}" # "Honeywell"
identifiers: "{{ device_unique_id }}" # "rtl_433_honeywell_123456"
suggested_area: "{{ device_area }}" # "Foyer"
- name: "Tamper"
object_id: "{{ device_name_safe + '_tamper' }}" # "front_door_sensor_tamper"
unique_id: "{{ device_unique_id + '_tamper' }}" # "rtl_433_honeywell_123456_tamper"
state_topic: "{{ device_topic + '/tamper' }}" # "rtl_433/+/devices/Honeywell-Security/8/123456/tamper"
payload_on: 1 # open
payload_off: 0 # closed
device_class: safety
force_update: true # Needed?
expire_after: "{{ device_unavailable_seconds }}"
device:
identifiers: "{{ device_unique_id }}"
# Event or Heartbeat, inverted to make event == True
# TODO - should this be considered a diagnostic entity?
- name: "Event"
object_id: "{{ device_name_safe + '_event' }}" # "front_door_sensor_event"
unique_id: "{{ device_unique_id + '_event' }}" #"rtl_433_honeywell_123456_event"
state_topic: "{{ device_topic + '/heartbeat' }}" # "rtl_433/+/devices/Honeywell-Security/8/123456/heartbeat"
payload_on: 0 # 0 == event
payload_off: 1 # 1 == heartbeat
force_update: true # Needed?
expire_after: "{{ device_unavailable_seconds }}"
device:
identifiers: "{{ device_unique_id }}"
sensor:
# Device undecoded status byte for diagnostic purposes
# Honeywell-Security misnamed as "event", should be renamed in decoder
- name: "Status"
object_id: "{{ device_name_safe + '_status' }}" # "front_door_sensor_status"
unique_id: "{{ device_unique_id + '_status' }}" # "rtl_433_honeywell_123456_event"
state_topic: "{{ device_topic + '/event' }}" # "rtl_433/+/devices/Honeywell-Security/8/123456/event"
force_update: true # Needed?
expire_after: "{{ device_unavailable_seconds }}"
entity_category: diagnostic
# todo eventually should be disabled by default?
device:
identifiers: "{{ device_unique_id }}"
# TODO Everything below this line are generic rtl_433 sensors
- name: "Battery"
object_id: "{{ device_name_safe + '_battery' }}" # "front_door_sensor_battery"
unique_id: "{{ device_unique_id + '_battery' }}" # "rtl_433_honeywell_123456_battery"
state_topic: "{{ device_topic + '/battery_ok' }}" # "rtl_433/+/devices/Honeywell-Security/8/123456/battery_ok"
value_template: "{{ '{{' }} float(value|int) * 99 + 1 }}"
device_class: battery
unit_of_measurement: "%"
force_update: true # Needed?
expire_after: "{{ device_unavailable_seconds }}"
entity_category: diagnostic
device:
identifiers: "{{ device_unique_id }}"
- name: "RSSI"
object_id: "{{ device_name_safe + '_rssi' }}" # "front_door_sensor_rssi"
unique_id: "{{ device_unique_id + '_rssi' }}" # "rtl_433_honeywell_123456_rssi"
state_topic: "{{ device_topic + '/rssi' }}" # "rtl_433/+/devices/Honeywell-Security/8/123456/rssi"
value_template: "{{ '{{' }} value|float|round(2) }}"
unit_of_measurement: "dB"
device_class: signal_strength
force_update: true # Needed?
expire_after: "{{ device_unavailable_seconds }}"
entity_category: diagnostic
# todo eventually should be disabled by default?
device:
identifiers: "{{ device_unique_id }}"
- name: "SNR"
object_id: "{{ device_name_safe + '_snr' }}" # "front_door_sensor_snr"
unique_id: "{{ device_unique_id + '_snr' }}" # "rtl_433_honeywell_123456_snr"
state_topic: "{{ device_topic + '/snr' }}" # "rtl_433/+/devices/Honeywell-Security/8/123456/snr"
value_template: "{{ '{{' }} value|float|round(2) }}"
unit_of_measurement: "dB"
device_class: signal_strength
force_update: true # Needed?
expire_after: "{{ device_unavailable_seconds }}"
entity_category: diagnostic
# todo eventually should be disabled by default?
device:
identifiers: "{{ device_unique_id }}"
- name: "noise"
object_id: "{{ device_name_safe + '_noise' }}" # "front_door_sensor_noise"
unique_id: "{{ device_unique_id + '_noise' }}" # "rtl_433_honeywell_123456_noise"
state_topic: "{{ device_topic + '/noise' }}" # "rtl_433/+/devices/Honeywell-Security/8/123456/noise"
value_template: "{{ '{{' }} value|float|round(2) }}"
unit_of_measurement: "dB"
device_class: signal_strength
force_update: true # Needed?
expire_after: "{{ device_unavailable_seconds }}"
entity_category: diagnostic
enabled_by_default: false
# todo eventually should be disabled by default?
device:
identifiers: "{{ device_unique_id }}"
Regarding the autodiscovery script in this repo (mqtt_hass), I had a note that for the Honeywell security sensors it was only generating binary_sensors for 'alarm' and 'tamper' fields. As I said above, the autodiscovery script does have definitions for contact_open, reed_open, but they don't seem to get processed. I never tried to figure out why.
very cool jinja2 stuff. can you post the command line to make a yaml from it (presumably you !include it). do you have to worry about having all mqtt in the same generated file?
Also, if you have wisdom about why you chose jinja2 instead of m4, I'd like to hear it. Yes, I know m4 is wicked old school.
I didn't post the script that runs Jinja2 because was a quick first pass at getting it working. It was good enough for me, but would be a bit embarrassing to post. I will put something together. I have some aspirations at building something to replace the autodiscovery stuff, but I get stuff that works well enough for me and don't get back to it.
As far as the code goes, it's one of those 98% scaffolding things. You load all the variables in a dict, and then call jinja2 to process the template. You get a string back and write it to a file. The interesting work is done in 2 lines.
The Home Assistant MQTT configuration doesn't have any real interdependencies. I like to have all the config for one sensor (or one integration) in one file. It works well with source code control. It also lets me fall back on old habits--If one sensor (on integration) needs to be disabled mv sensor1.yaml sensor1.yaml-
Yes, I know m4 from the old days, particularly when I wound up responsible for all the sendmail configs for my company because I was the one that did the internet firewall.
For this Jinja2 really makes sense because it is what Home Assistant uses and you have to climb that learning curve for Home Assistant (especially if you don't want to be an appliance operator). So anything I did there would be eaiser for Hass users to pick up. I've also adopted Ansible for configuration management, which uses YAML + Jinja2 templating.
Another bonus, if you have a Home Assistant install handy, you have easy access to a sort of REPL for Jinja2 in Developer Tools -> Templates.
Until I get around to cleaning up my script, see this blog & code I found for something close in terms of using Jinja2 to generate config files:
- Blog post Configuration generator with python and Jinja2
- Code that reads JSON parameters and a template: https://github.com/hoelsner/python-script-examples/blob/master/config-generator-with-python-and-jinja2/json_based_config_generator.py
This looks like an updated version of the Jinja2 tutorial I found a few years back. It was also generating some router config files.
Same repo, this looks useful for defining custom Jinja2 functions (filters): This could be useful for simplifying the templates.
- Custom filters for a Jinja2 based config generator
- Code: https://github.com/hoelsner/python-script-examples/tree/master/config-generator-with-custom-filters
@rct that's a cool approach, thanks for the tips. What does the script look like to run populate the config from your sensors list yml (and how do you run it)?
Thanks @gdt for your feedback as well!
@digitaltopo I am also using manual config, more because I started that way, but I think @rct's approach is wise given the ability to customize.
Thanks for the pointers; that helps enough to point me on the right track. Writing a bit of code is fine, but I wasn't sure if there was already a program. My remaining worry is that I'd like (probably) to have some things in regular config and some generated. It seems like HA is not ok with mqtt:/sensor: in one place and in another, unioning the included lists/maps. But that's an HA problem, not an rtl_433 problem.
@digitaltopo - the script can be run anywhere that Python is installed with the YAML and Jinja2 modules, so windows, Linux, Mac.
I think you should be able run it on Home Assistant using Frenck's ssh/web terminal add-on (the community ssh, not the core one). Then you wouldn't have to copy any files over.
The add-on's Python doesn't have the Jinja2 module installed by default, but it could be added with apk add py3-jinja2
If I turn this into a little more of an app, rather than go the add-on route, I'd probably make it an AppDaemon package. AppDaemon has somewhat fallen out of favor as functionality in Home Assistant has increased. But AppDaemon would take care of a bit of the scaffolding and provide a sort of UI. I've got a toy app that subscribes to rtl_433 via MQTT and and publishing some stats to help me track the health of the RF environment.
It also occurred to me that PyScript (which I haven't used yet) could be another way to integrate something like this. I don't think it is sandboxed from the filesystem. You could define a service in PyScript and then use Home Assistant's developer tools to call it.
Thanks for the pointers; that helps enough to point me on the right track. Writing a bit of code is fine, but I wasn't sure if there was already a program.
Give me a bit, I'll put what I have up in a scratch repo as a PoC.
It seems like HA is not ok with mqtt:/sensor: in one place and in another, unioning the included lists/maps. But that's an HA problem, not an rtl_433 problem.
Ah, it sounds like you are tripping over Home Assistant's various !include_* directives and/or hitting the issue where those YAML blocks need a distinct key because they are going to get merged in a big dict (or whatever key value structure).
When I tried to follow Frenck's split config file guidance from a number of years ago I kept banging my head against stuff like that. It used a merged list for sensors.
sensor: !include_dir_list ../entities/sensors
binary_sensor: !include_dir_list ../entities/binary_sensors
I gave up on that and just use packages which avoids (most if not all) of the conflicts. This is also from Frenck's split config, but now I just put stuff in a file in integrations and don't use the entitles lists.
homeassistant:
# load packages
packages: !include_dir_named integrations
I think some of the changes they've made in the past few years that resulted in some legacy config styles (like template sensors) changing which domain things are defined under in which order have improved things. In the past I sometimes needed to stick in some throwaway identifier like sensor foo_package:
@gdt @digitaltopo - your interest/questions motivated me to put my script up in a repo as a proof of concept:
- https://github.com/rct/hass-mqtt-templates
Regarding the autodiscovery script in this repo (mqtt_hass), I had a note that for the Honeywell security sensors it was only generating binary_sensors for 'alarm' and 'tamper' fields. As I said above, the autodiscovery script does have definitions for contact_open, reed_open, but they don't seem to get processed. I never tried to figure out why.
Do you think that this will get fixed at some point? I'm just setting some of these up and there is no open/closed state provided to the HA device.
So far it does not look like anybody is really trying to fix this. if you are willing to dig into python and HA, then I would guess it is not too hard to fix. From rct's comment, it sounds like the desired sensors look like they are in the autodiscovery payload, but HA seems not to pick them up. Perhaps you looking at them carefully will result in realizing why, or perhaps you could add lots of print tracing to the mqtt discovery processing code in HA.
I'm far from the correct person to be fixing code! I will actually dig around a little and see what I see. Thanks.
For all the sensors that are decoded by the Honeywell 5800 series decoder, there is no single definition for state that will cover all of the cases.
Like with a physical Honeywell panel you need to configure which bit (contact, reed, alarm) contains the state you care about.
A number of these sensors can cover 2 or more zones. For example you can use both the reed switch and a hardwired contact.
I'm a bit surprised that those states (reed open, contact open, state) aren't included as attributes in the Home Assistant sensor that's created in the auto-discovery add-on, instead. I'm just a novice, so I don't know if there's a technical reason for that. The door/window sensors I'm using do in fact have the dual contact types (reed and magnetic).
I did attempt to look through the files to see if I could figure out a way to make that happen, but that's beyond my skill set unfortunately.