fritzinfluxdb
fritzinfluxdb copied to clipboard
bb-Ricardo
Include Smart Home / DECT devices?
Works surprisingly well in a FreeBSD jail (TrueNAS 13) with pm2 as process manager. Ask away if somebody has questions regarding to it.
Question for the developer: Smart Home devices like FRITZ!DECT 200 can provide useful data such as temperature readings. Any way to get to that data through your python script?
Hi,
Due to the lack of a Fritz!DECT Smart home device I can't get the readings and implement the measurements.
Can you post the Lua API output here?
You need to browser to the page of the FritzBox where the values are shown. Then open your browsers developer tools and get the full json output of the data.lua endpoint. This would help a lot.
Thank you.
Sorry, I tried to find that data.lua but didn't find it anywhere. Isn't there a .lua link to call dect devices?
Something like this: http://fritz.box/webservices/homeautoswitch.lua?sid=[SesionID]&switchcmd=getdevicelistinfos
However, this wouldn't work for me and I don't know why. Edit: Ok I've found out why, because my login sid is always 0 (0000000000000000) as per fritz.box/login_sid.lua which is according to AVM docs invalid. So I'm stuck here...
Ahh ok, for this data it seems to be homeautoswitch.lua. Did you use the Browsers developer tools to retrieve the data?
I actually can't get to the desired data, because I can't get a valid session id, like I mentioned in the above comment. There is no way to access any lua api data, w/o a valid sid. Mine is always reverted back to 000...., which is invalid.
I may have already accomplished what you are trying to do using the TR-069 protocol.
For each device I added the following to classes/fritzbox/services_tr069.py (NewIndex: 0-3, home prefix: 0-3):
{
"name": "X_AVM-DE_Homeauto",
"interval": 120,
"actions": [{
"name": "GetGenericDeviceInfos",
"params": {
"NewIndex": 0
}
}],
"value_instances": {
"NewAIN": "home0_ain",
"NewDeviceName": "home0_devicename",
"NewPresent": "home0_present",
"NewTemperatureIsEnabled": "home0_temp_enabled",
"NewTemperatureIsValid": "home0_temp_valid",
"NewTemperatureCelsius": "home0_temp_celsius",
"NewTemperatureOffset": "home0_temp_offset",
"NewMultimeterIsEnabled": "home0_multimeter_enabled",
"NewMultimeterIsValid": "home0_multimeter_valid",
"NewMultimeterPower": "home0_multimeter_power",
"NewMultimeterEnergy": "home0_multimeter_energy",
"NewHkrIsEnabled": "home0_heating_enabled",
"NewHkrIsValid": "home0_heating_valid",
"NewHkrIsTemperature": "home0_heating_istemperature",
"NewHkrSetVentilStatus": "home0_heating_setventil",
"NewHkrSetTemperature": "home0_heating_settemperature",
"NewHkrSetTemperature": "home0_heating_settemperature",
"NewHkrReduceVentilStatus": "home0_heating_reduceventil",
"NewHkrReduceTemperature": "home0_heating_reducetemperature",
"NewHkrComfortVentilStatus": "home0_heating_comfortventil",
"NewHkrComfortTemperature": "home0_heating_comforttemperature"
}
}
My Flux query joins home{index}_devicename with home{index}_temp_celsius and creates this using pivot:
But if you just ignore the home{index}_devicename and add names statically, you can achieve this easier, I guess.
@matthiasstock Hi, this already looks promising. Are these values are also available via the web interface?
Are you able to check if this data can be extracted via homeautoswitch.lua in JSON?
Tank you.
I am able to query the temperatures using this endpoint.
curl -s -H 'Accept: application/json' "http://fritz.box/webservices/homeautoswitch.lua?sid=${mysid}&switchcmd=getdevicelistinfos"
However, I get XML instead of JSON. Does this already help or what else should I try?
<?xml version="1.0"?>
<devicelist version="1" fwversion="7.29">
<device identifier="09995 0733992" id="16" functionbitmask="320" fwversion="05.02" manufacturer="AVM" productname="FRITZ!DECT 301">
<present>1</present>
<txbusy>0</txbusy>
<name>Zimmer 1</name>
<battery>70</battery>
<batterylow>0</batterylow>
<temperature>
<celsius>260</celsius>
<offset>0</offset>
</temperature>
<hkr>
<tist>52</tist>
<tsoll>253</tsoll>
<absenk>38</absenk>
<komfort>42</komfort>
<lock>1</lock>
<devicelock>0</devicelock>
<errorcode>0</errorcode>
<windowopenactiv>0</windowopenactiv>
<windowopenactiveendtime>0</windowopenactiveendtime>
<boostactive>0</boostactive>
<boostactiveendtime>0</boostactiveendtime>
<batterylow>0</batterylow>
<battery>70</battery>
<nextchange>
<endperiod>1661979600</endperiod>
<tchange>38</tchange>
</nextchange>
<summeractive>1</summeractive>
<holidayactive>0</holidayactive>
</hkr>
</device>
<device identifier="09995 0733974" id="17" functionbitmask="320" fwversion="05.02" manufacturer="AVM" productname="FRITZ!DECT 301">
<present>1</present>
<txbusy>0</txbusy>
<name>Zimmer 2</name>
<battery>70</battery>
<batterylow>0</batterylow>
<temperature>
<celsius>270</celsius>
<offset>0</offset>
</temperature>
<hkr>
<tist>54</tist>
<tsoll>253</tsoll>
<absenk>38</absenk>
<komfort>42</komfort>
<lock>1</lock>
<devicelock>0</devicelock>
<errorcode>0</errorcode>
<windowopenactiv>0</windowopenactiv>
<windowopenactiveendtime>0</windowopenactiveendtime>
<boostactive>0</boostactive>
<boostactiveendtime>0</boostactiveendtime>
<batterylow>0</batterylow>
<battery>70</battery>
<nextchange>
<endperiod>1661979600</endperiod>
<tchange>38</tchange>
</nextchange>
<summeractive>1</summeractive>
<holidayactive>0</holidayactive>
</hkr>
</device>
<device identifier="11630 0248598" id="18" functionbitmask="35712" fwversion="04.16" manufacturer="AVM" productname="FRITZ!DECT 200">
<present>1</present>
<txbusy>0</txbusy>
<name>K8s</name>
<switch>
<state>0</state>
<mode>manuell</mode>
<lock>0</lock>
<devicelock>0</devicelock>
</switch>
<simpleonoff>
<state>0</state>
</simpleonoff>
<powermeter>
<voltage>226873</voltage>
<power>0</power>
<energy>434</energy>
</powermeter>
<temperature>
<celsius>290</celsius>
<offset>0</offset>
</temperature>
</device>
<device identifier="11657 0271380" id="19" functionbitmask="35712" fwversion="04.17" manufacturer="AVM" productname="FRITZ!DECT 210">
<present>1</present>
<txbusy>0</txbusy>
<name>Schuppen</name>
<switch>
<state>0</state>
<mode>auto</mode>
<lock>0</lock>
<devicelock>0</devicelock>
</switch>
<simpleonoff>
<state>0</state>
</simpleonoff>
<powermeter>
<voltage>223379</voltage>
<power>0</power>
<energy>107357</energy>
</powermeter>
<temperature>
<celsius>245</celsius>
<offset>0</offset>
</temperature>
</device>
</devicelist>
Hi,
well, this is not really what I expected. But a simple google search brought this: https://avm.de/fileadmin/user_upload/Global/Service/Schnittstellen/AHA-HTTP-Interface.pdf
Not sure If I'm willing to implement yet another Interface.
I guess this is in popular demand. Let's see if we can implement this in the next few weeks.
Hey @SebastianChudoba, @matthiasstock, @4920441, @donnerhund,
I added the first version of support for homautomation values. Can you please checkout branch add-homautomation and let me know if if home automation values are read and written to InfluxDB? Just add -v and you will see what's written not InfluxDB.
Thank you
Hi Ricardo, Thank you very much for the quick implementation of this function. I have just started the first attempt, but I get the following error: Error: FritzBoxLuaService 'System Stats' instance has unsupported url_path: None
The following should fix the url_path: None problem, but the branch still doesn't work for me, yet. Maybe I'll have time to look at it this weekend.
diff --git a/fritzinfluxdb/classes/fritzbox/service_handler.py b/fritzinfluxdb/classes/fritzbox/service_handler.py
index a45d508..38644fd 100644
--- a/fritzinfluxdb/classes/fritzbox/service_handler.py
+++ b/fritzinfluxdb/classes/fritzbox/service_handler.py
@@ -154,7 +154,7 @@ class FritzBoxLuaService(FritzBoxService):
super().__init__(service_data)
- url_path = service_data.get("url_path")
+ url_path = service_data.get("url_path") or FritzBoxLuaURLPath.data
if url_path == FritzBoxLuaURLPath.data:
self.page = service_data.get("page")
My PR #50 fixes my requests, but leads to this error:
2022-09-16 17:17:25,566 - ERROR: FritzBox Lua returned: 403 : Forbidden
2022-09-16 17:17:25,566 - ERROR: FritzBox Lua returned body: None
2022-09-16 17:17:25,567 - ERROR: Unable to request FritzBox Lua service 'Home Automation', no data returned
2022-09-16 17:17:25,567 - INFO: FritzBox Lua service 'Home Automation' will be disabled.
I have changed @matthiasstock's change "by hand" in the corresponding file. After starting ./fritzinfluxdb.py -v this time there are various DEBUG lines, in between the following error lines. I use a Fritbox 7590 with the following firmware: 07.39-97058 BETA
2022-09-16 17:53:03,522 - ERROR: FritzBox Lua returned: 404 : Not Found 2022-09-16 17:53:03,522 - ERROR: FritzBox Lua returned body: None 2022-09-16 17:53:03,522 - ERROR: Unable to request FritzBox Lua service 'System Stats', no data returned 2022-09-16 17:53:03,522 - INFO: FritzBox Lua service 'System Stats' will be disabled. 2022-09-16 17:53:03,530 - ERROR: FritzBox Lua returned: 404 : Not Found 2022-09-16 17:53:03,530 - ERROR: FritzBox Lua returned body: None 2022-09-16 17:53:03,530 - ERROR: Unable to request FritzBox Lua service 'Energy Stats', no data returned 2022-09-16 17:53:03,530 - INFO: FritzBox Lua service 'Energy Stats' will be disabled. 2022-09-16 17:53:03,530 - DEBUG: FritzOS version 7.39 not in list of supported versions for System logs: ['7.29', '7.30', '7.31'] 2022-09-16 17:53:03,538 - ERROR: FritzBox Lua returned: 404 : Not Found 2022-09-16 17:53:03,538 - ERROR: FritzBox Lua returned body: None 2022-09-16 17:53:03,538 - ERROR: Unable to request FritzBox Lua service 'System logs', no data returned 2022-09-16 17:53:03,538 - INFO: FritzBox Lua service 'System logs' will be disabled. 2022-09-16 17:53:03,538 - DEBUG: FritzOS version 7.39 not in list of supported versions for Internet connection logs: ['7.29', '7.30', '7.31'] 2022-09-16 17:53:03,546 - ERROR: FritzBox Lua returned: 404 : Not Found 2022-09-16 17:53:03,546 - ERROR: FritzBox Lua returned body: None 2022-09-16 17:53:03,546 - ERROR: Unable to request FritzBox Lua service 'Internet connection logs', no data returned 2022-09-16 17:53:03,546 - INFO: FritzBox Lua service 'Internet connection logs' will be disabled. 2022-09-16 17:53:03,546 - DEBUG: FritzOS version 7.39 not in list of supported versions for Telephony logs: ['7.29', '7.30', '7.31'] 2022-09-16 17:53:03,554 - ERROR: FritzBox Lua returned: 404 : Not Found 2022-09-16 17:53:03,554 - ERROR: FritzBox Lua returned body: None 2022-09-16 17:53:03,554 - ERROR: Unable to request FritzBox Lua service 'Telephony logs', no data returned 2022-09-16 17:53:03,554 - INFO: FritzBox Lua service 'Telephony logs' will be disabled. 2022-09-16 17:53:03,554 - DEBUG: FritzOS version 7.39 not in list of supported versions for WLAN logs: ['7.29', '7.30', '7.31'] 2022-09-16 17:53:03,562 - ERROR: FritzBox Lua returned: 404 : Not Found 2022-09-16 17:53:03,562 - ERROR: FritzBox Lua returned body: None 2022-09-16 17:53:03,562 - ERROR: Unable to request FritzBox Lua service 'WLAN logs', no data returned 2022-09-16 17:53:03,562 - INFO: FritzBox Lua service 'WLAN logs' will be disabled. 2022-09-16 17:53:03,562 - DEBUG: FritzOS version 7.39 not in list of supported versions for USB Devices logs: ['7.29', '7.30', '7.31'] 2022-09-16 17:53:03,606 - ERROR: FritzBox Lua returned: 404 : Not Found 2022-09-16 17:53:03,606 - ERROR: FritzBox Lua returned body: None 2022-09-16 17:53:03,606 - ERROR: Unable to request FritzBox Lua service 'USB Devices logs', no data returned 2022-09-16 17:53:03,606 - INFO: FritzBox Lua service 'USB Devices logs' will be disabled. 2022-09-16 17:53:03,614 - ERROR: FritzBox Lua returned: 404 : Not Found 2022-09-16 17:53:03,614 - ERROR: FritzBox Lua returned body: None 2022-09-16 17:53:03,614 - ERROR: Unable to request FritzBox Lua service 'Active network hosts', no data returned 2022-09-16 17:53:03,614 - INFO: FritzBox Lua service 'Active network hosts' will be disabled. 2022-09-16 17:53:03,622 - ERROR: FritzBox Lua returned: 404 : Not Found 2022-09-16 17:53:03,623 - ERROR: FritzBox Lua returned body: None 2022-09-16 17:53:03,623 - ERROR: Unable to request FritzBox Lua service 'Passive network hosts', no data returned 2022-09-16 17:53:03,623 - INFO: FritzBox Lua service 'Passive network hosts' will be disabled. 2022-09-16 17:53:03,623 - DEBUG: FritzOS version 7.39 not in list of supported versions for VPN Users: ['7.29', '7.30', '7.31'] 2022-09-16 17:53:03,630 - ERROR: FritzBox Lua returned: 404 : Not Found 2022-09-16 17:53:03,630 - ERROR: FritzBox Lua returned body: None 2022-09-16 17:53:03,630 - ERROR: Unable to request FritzBox Lua service 'VPN Users', no data returned 2022-09-16 17:53:03,630 - INFO: FritzBox Lua service 'VPN Users' will be disabled. 2022-09-16 17:53:03,639 - ERROR: FritzBox Lua returned: 404 : Not Found 2022-09-16 17:53:03,639 - ERROR: FritzBox Lua returned body: None 2022-09-16 17:53:03,639 - ERROR: Unable to request FritzBox Lua service 'DSL Info', no data returned 2022-09-16 17:53:03,639 - INFO: FritzBox Lua service 'DSL Info' will be disabled. 2022-09-16 17:53:03,649 - ERROR: FritzBox Lua returned: 404 : Not Found 2022-09-16 17:53:03,649 - ERROR: FritzBox Lua returned body: None 2022-09-16 17:53:03,649 - ERROR: Unable to request FritzBox Lua service 'Cable Info', no data returned 2022-09-16 17:53:03,649 - INFO: FritzBox Lua service 'Cable Info' will be disabled. 2022-09-16 17:53:03,665 - ERROR: FritzBox Lua returned: 404 : Not Found 2022-09-16 17:53:03,665 - ERROR: FritzBox Lua returned body: None 2022-09-16 17:53:03,666 - ERROR: Unable to request FritzBox Lua service 'Home Automation', no data returned 2022-09-16 17:53:03,666 - INFO: FritzBox Lua service 'Home Automation' will be disabled. 2022-09-16 17:53:03,682 - DEBUG: Successfully wrote 75 measurements to InfluxDB 2022-09-16 17:53:03,682 - DEBUG: Current InfluxDB measurement buffer length: 0 2022-09-16 17:53:04,684 - DEBUG: InfluxDB data queue: No measurements found in queue 2022-09-16 17:53:04,684 - DEBUG: Current InfluxDB measurement buffer length: 0
damn, It's kinda difficult to write this stuff 'blind' without any actual real devices.
But I will have a look tonight. I have an idea. I must have implemented the url_path incorrectly.
if you run it with -vv you should also see the HTTP requests.
can you guys pull the latest add-homautomation branch and try again please?
thank you
With another fix in PR #52, this branch works for me and writes home automation data to InfluxDB.
Please note that I just took a random look at the data.
https://github.com/bb-Ricardo/fritzinfluxdb/pull/52#issuecomment-1249827586:
It's great that we get the data but I have no idea right now how to visualise for example the heating data for the thermostats. Any idea? should we move this to discussions?
I am going to start collecting data from the add-homautomation branch to another bucket this weekend. I am also interested in using this data instead of my own approach.
Anyway, I can't promise that I'll find the time to enhance the dashboard within this month, but I'll try.
At first glance, the data from my two FRITZ!DECT devices arrive in InfluxDB :)
Now it's time to adapt the previously used Grafana Dashboard...
Nice project! Thanks because that is exactly what I was looking for. I own some of these temperature sensors and tested your code. I would like to suggest some additions to services_lua_homeauto.py
- With the
namefield you can identify the single DECT301 devices with the room name you configured them with - Additional to temperature the measured and offset values are interesting when you fine tune the settings: temperature.celsius is usually the value directly measured by the senor. When radiator is on the measurement will not be the same as room temperature and always be higher. Therefore you have an offset to make some corrections. Otherwise DECT301 device will turn heating down. Caluclated temperature field should be close to room temperature. It looks like Fritz calculates this value themselves and writes it into
tistfield
"ha_name": {
"data_path": "devicelist.device",
"type": list,
"next": {
# data struct type: dict
"type": str,
"tags_function": lambda data: {"name": data.get("name")},
"data_path": "name"
},
"exclude_filter_function": lambda data: "device" not in data.get("devicelist").keys()
},
"ha_id": {
"data_path": "devicelist.device",
"type": list,
"next": {
# data struct type: dict
"type": str,
"tags_function": lambda data: {"name": data.get("name")},
"data_path": "@id"
},
"exclude_filter_function": lambda data: "device" not in data.get("devicelist").keys()
},
"ha_identifier": {
"data_path": "devicelist.device",
"type": list,
"next": {
# data struct type: dict
"type": str,
"tags_function": lambda data: {"name": data.get("name")},
"data_path": "@identifier"
},
"exclude_filter_function": lambda data: "device" not in data.get("devicelist").keys()
},
"ha_temperature_sensor": {
"data_path": "devicelist.device",
"type": list,
"next": {
# data struct type: dict
"type": float,
"tags_function": lambda data: {"name": data.get("name")},
"value_function": lambda data: (
float((int(grab(data, "temperature.celsius")))/10)
),
"exclude_filter_function": lambda data: "temperature" not in data.keys()
},
"exclude_filter_function": lambda data: "device" not in data.get("devicelist").keys()
},
"ha_temperature_offset": {
"data_path": "devicelist.device",
"type": list,
"next": {
# data struct type: dict
"type": float,
"tags_function": lambda data: {"name": data.get("name")},
"value_function": lambda data: (
float((int(grab(data, "temperature.offset")))/10)
),
"exclude_filter_function": lambda data: "temperature" not in data.keys()
},
"exclude_filter_function": lambda data: "device" not in data.get("devicelist").keys()
},
Here are some screenshots from my instance running the branch (room names are directly used from the name field content)
During my test I got some error message when a DECT301 device temporary loses connection to FritzBox (could not find out why this sometimes happens so far). It seems that then some values are delivered back with "None" instead of 0:
2022-09-17 12:26:38,411 - ERROR: Unable to extract 'None' form '{ '@identifier': '09995 0906197', '@id': '16', '@functionbitmask': '320', '@fwversion': '05.02', '@manufacturer': 'AVM', '@productname': 'FRITZ!DECT 301', 'present': '0', 'txbusy': '0', 'name': 'Arbeitszimmer', 'battery': '60', 'batterylow': '0', 'temperature': {'celsius': None, 'offset': None}, 'hkr': {'tist': None, 'tsoll': None, 'absenk': None, 'komfort': None, 'lock': None, 'devicelock': None, 'errorcode': '0', 'windowopenactiv': '0', 'windowopenactiveendtime': '0', 'boostactive': '0', 'boostactiveendtime': '0', 'batterylow': '0', 'battery': '0', 'nextchange': {'endperiod': '0', 'tchange': '255'}, 'summeractive': None, 'holidayactive': None}}', got '<class 'NoneType'>'
Thank you for the testing. From I quick glace I didn't get the name and offset part. Will have a look later
The implemented temperature value is completely correct: (temperature.celsius + temperature.offset)/10. I just wanted to say that it will be useful also writing the separate values of temperature.celsius/10 and temperature.offset/10 to indfluxdb as well
Hi @schrdh,
I just pushed a new version to next-release including your suggestions regarding temperature and a fix for the ERROR messages.
Can you have a look if it behaves better now?
thank you