localtuya
localtuya copied to clipboard
rospogrigio
Datapoint reports rawdata, needs to be energy information (phase_a_ + current, voltage, power) - SOLVED
The problem
Device: TUYA APP 1P 40A Rail Din WIFI compteur d'énergie intelligent consommation d'énergie kWh interrupteur de disjoncteur (link: https://www.amazon.fr/dp/B09FPGNYFK/ref=pe_27091421_487030221_TE_SCE_dp_1)
When I try to get data in localtuya, the power consumption (which works fine in Tuya-cloud) comes in as rawdata. I think the string is a Base64-coded value, but I do not known how to convert this to current, voltage and power.
This is the reading I get on datapoint 6 (it comes as rawdata): CWYAAAAAAAA= x This DP is described in the Tuya Cloud as "phase_a". Should this value be converted into an array (because I saw this in other devices)? How?
Can anybody help pls? Thank you.
UPDATE - 11/01/2023
I have found a solution thanks to @MaximumSU for this device, so I can deconnect completely from TuyaCloud.
Step 1 - INSTALL YOUR DEVICE IN LOCALTUYA:
- add the device in localtuya as usual, but manually add datapoint 6 (this will result in a base64-encoded string, which containes Voltage, Current and Power)
- make sure your device is getting readings in this variable (...phase_a): ex. "CTgAAAAAAAA="
Step 2 - ACTIVATE THE PYTHON BASE64-decode SCRIPT:
- make sure pyscript is installed
- check configuration.yaml for these settings:
pyscript:
allow_all_imports: true
hass_is_global: true
- create a script named base64_int.py, which is located in
/pyscript (you can use VSCode addon) - copy this code in your base64_int.py (you can rename your sensors and functions, but stay consequent and change everywhere!!)
@state_trigger('sensor.boiler_lavande_phase_a')
def base64_decode_sensor_lavande():
#!/usr/bin/env python3
import base64
import sys
# Decoding value from datapoint 6 which is base64-based (manually activated)
value_phase_a = state.get('sensor.boiler_lavande_phase_a')
value_decoded = base64.b64decode(value_phase_a)
# Calculating Voltage from base64-decoded value
value_voltage = round(int.from_bytes(value_decoded[0:2], byteorder='big')*0.1, 1)
state.set('sensor.boiler_lavande_voltage_py', value_voltage)
# Calculating Current from base64-decoded value
value_current = round(int.from_bytes(value_decoded[3:5], byteorder='big')*0.001, 2)
state.set('sensor.boiler_lavande_current_py', value_current)
# Calculating Power from base64-decoded value
value_power = round(int.from_bytes(value_decoded[6:8], byteorder='big')*0.001, 3)
state.set('sensor.boiler_lavande_power_py', value_power)
Step 3 - PREDEFINE YOUR SENSORS (maybe this needs improvement):
- edit your
/.storage/core.entity_registry with VSCode - paste these sensor-definitions after your current entities for this device (rename where necessary)
{
"aliases": [],
"area_id": null,
"capabilities": {
"state_class": "measurement"
},
"config_entry_id": "2e3c6d8ffb3712bdc356c18de5aeb41e",
"device_class": null,
"device_id": "ed2f8541d843b59ce8ef7fb22d4c54e2",
"disabled_by": null,
"entity_category": null,
"entity_id": "sensor.boiler_lavande_current_py",
"hidden_by": null,
"icon": null,
"id": "3b68d9cdddbd967a1fba6a52e531538ca",
"has_entity_name": false,
"name": null,
"options": {},
"original_device_class": "current",
"original_icon": "mdi:current-ac",
"original_name": "Boiler (Lavande) - Phase A - ampère",
"platform": "localtuya",
"supported_features": 0,
"translation_key": null,
"unique_id": "local_bf69893b9f9e8b7ee08vuu_6amp",
"unit_of_measurement": "A"
},
{
"aliases": [],
"area_id": null,
"capabilities": {
"state_class": "measurement"
},
"config_entry_id": "2e3c6d8ffb3712bdc356c18de5aeb41e",
"device_class": null,
"device_id": "ed2f8541d843b59ce8ef7fb22d4c54e2",
"disabled_by": null,
"entity_category": null,
"entity_id": "sensor.boiler_lavande_voltage_py",
"hidden_by": null,
"icon": null,
"id": "3b68d9cdddbd967a1fba6a52e531538ca",
"has_entity_name": false,
"name": null,
"options": {},
"original_device_class": "voltage",
"original_icon": "mdi:sine-wave",
"original_name": "Boiler (Lavande) - Phase A - voltage",
"platform": "localtuya",
"supported_features": 0,
"translation_key": null,
"unique_id": "local_bf69893b9f9e8b7ee08vuu_6vol",
"unit_of_measurement": "V"
},
{
"aliases": [],
"area_id": null,
"capabilities": {
"state_class": "measurement"
},
"config_entry_id": "2e3c6d8ffb3712bdc356c18de5aeb41e",
"device_class": null,
"device_id": "ed2f8541d843b59ce8ef7fb22d4c54e2",
"disabled_by": null,
"entity_category": null,
"entity_id": "sensor.boiler_lavande_power_py",
"hidden_by": null,
"icon": null,
"id": "3b68d9cdddbd967a1fba6a52e531538ca",
"has_entity_name": false,
"name": null,
"options": {},
"original_device_class": "power",
"original_icon": "mdi:lightning-bolt",
"original_name": "Boiler (Lavande) - Phase A - power",
"platform": "localtuya",
"supported_features": 0,
"translation_key": null,
"unique_id": "local_bf69893b9f9e8b7ee08vuu_6pow",
"unit_of_measurement": "kW"
},
Enjoy :-)
Environment
- Localtuya version: 4.1.1
- Home Assistant Core version: Home Assistant OS 9.4, version: 2022.12.8
- [x] Are you using the Home Assistant Tuya Cloud component ?
- [x] Are you using the Tuya App in parallel ?
Configuration configuration.yaml or config_flow
"bf69893b9f9e8b7ee08vuu": {
"friendly_name": "Tuya disjoncteur/compteur DIN 40A",
"local_key": "c36...a09",
"host": "192.168.0.222",
"device_id": "bf69893b9f9e8b7ee08vuu",
"protocol_version": "3.3",
"manual_dps_strings": "6",
"model": "breaker",
"dps_strings": [
"1 (value: 6747)",
"9 (value: 0)",
"11 (value: False)",
"12 (value: False)",
"13 (value: 0)",
"14 (value: 0)",
"15 (value: 0)",
"16 (value: True)",
"19 (value: 4567325321)",
"21 (value: False)",
"6 (value: -1)"
],
"entities": [
{
"id": 16,
"friendly_name": "Boiler (Lavande)",
"restore_on_reconnect": false,
"is_passive_entity": false,
"platform": "switch"
},
{
"id": 1,
"friendly_name": "Boiler (Lavande) - energieverbruik",
"unit_of_measurement": "kWh",
"device_class": "power",
"scaling": 0.01,
"platform": "sensor"
},
{
"id": 6,
"friendly_name": "Boiler (Lavande) - Base64-encoded raw data",
"unit_of_measurement": "x",
"device_class": "apparent_power",
"scaling": 1.0,
"platform": "sensor"
}
],
"product_key": "hz2o2tnezttmmx15"
DP dump
I added the DP6 manually while adding the device in localtuya because it wasn't detected automaticly.
total_forward_energy | Integer | { "unit": "kW·h", "min": 0, "max": 99999999, "scale": 2, "step": 1 }
-- | -- | --
phase_a | Raw | {}
fault | Bitmap | { "label": [ "short_circuit_alarm", "surge_alarm", "overload_alarm", "leakagecurr_alarm", "temp_dif_fault", "fire_alarm", "high_power_alarm", "self_test_alarm", "ov_cr", "unbalance_alarm", "ov_vol", "undervoltage_alarm", "miss_phase_alarm", "outage_alarm", "magnetism_alarm", "credit_alarm", "no_balance_alarm" ] }
switch_prepayment | Boolean | "{true,false}"
energy_reset | Enum | { "range": [ "empty" ] }
balance_energy | Integer | { "unit": "kW·h", "min": 0, "max": 99999999, "scale": 2, "step": 1 }
charge_energy | Integer | { "unit": "kW·h", "min": 0, "max": 999999, "scale": 2, "step": 1 }
leakage_current | Integer | { "unit": "mA", "min": 0, "max": 1000, "scale": 0, "step": 1 }
switch | Boolean | "{true,false}"
alarm_set_1 | Raw | {}
alarm_set_2 | Raw | {}
breaker_number | String | { "maxlen": 255 }
leakagecurr_test | Boolean | "{true,false}"
Have the same issue with 3 phase socket breaker with meetering capability. It report Voltage, Current and another values of 3 phases in one DP as base64 encoded bites (2 byte+2byte+2byte for each A,B,C phase). Your decoded value is 240.6 0 0 by this method. Theese DP is not reported by device itself nor in tuya iot API. Sadly, this functionality (DP values binary decoder) is not realised yet in localtuya.
@MaximumSU how did you decode this string 'CWYAAAAAAAA=' , I tried base64_decode in python but it didn't give me any values, only an x,f,m,...
@BartGysens There are 4 2xByte values in your payload. Decode all string with base64 then take first (second, third, fourth) two bytes and convert it with int.from_bytes(state, byteorder='big'), then multiple by 0.1 - you taked the final value I created the fork of this project and added this functionality here
@MaximumSU thank you for talking me through this!
This is my code and something is not right... It gives me a Voltage = 1388498.3 where it should be 236
`@state_trigger('sensor.boiler_lavande_phase_a') def base64_decode_sensor_lavande(): #!/usr/bin/env python3 import base64 import sys
value_phase_a = state.get('sensor.boiler_lavande_phase_a')
#state.set('sensor.boiler_lavande_voltage_script', value_phase_a)
value_decoded = base64.b64decode(value_phase_a).hex()
#state.set('sensor.boiler_lavande_voltage_script', value_decoded[0:4])
value_voltage = base64.b64decode(value_decoded[0:4])
#state.set('sensor.boiler_lavande_voltage_script', value_voltage)
value_voltage = int.from_bytes( base64.b64decode(value_decoded[0:4]), 'big')*0.1
state.set('sensor.boiler_lavande_voltage_script', value_voltage)`
@BartGysens ".hex" not need (it produces hexadecimal string, but you need the binary string of bytes, it is already here) and 2 bytes ([0:2]), not 3 ([0:4]) See below:
value_phase_a = state.get('sensor.boiler_lavande_phase_a')
value_voltage = round(int.from_bytes(base64.b64decode(value_phase_a)[0:2], byteorder='big')*0.1)
state.set('sensor.boiler_lavande_voltage_script', value_voltage)
Hi, I've used your solution for my DDS238-2 meter made by OXT, but my data points were different, Now I have one problem, for my power value in the app I get positive power when getting power from the grid and a negative value when my solar is sending power to grid, but the value from the script is always positive. How can I correct that? here is the code:
@state_trigger(‘sensor.licznik_pradu_dol_fazaa’) def base64_decode_sensor_licznik(): #!/usr/bin/env python3 import base64 import sys
value_decoded = base64.b64decode(value_phase_a)
value_voltage = round(int.from_bytes(value_decoded[13:15], byteorder='big')*0.1, 3)
state.set('sensor.licznik_pradu_dol_voltage_py', value_voltage)
value_current = round(int.from_bytes(value_decoded[11:13], byteorder='big')*0.001, 3)
state.set('sensor.licznik_pradu_dol_current_py', value_current)
value_power = round(int.from_bytes(value_decoded[2:4], byteorder='big')*0.001, 3)
state.set('sensor.licznik_pradu_dol_power_py', value_power)
value_kvar = round(int.from_bytes(value_decoded[6:8], byteorder='big')*0.001, 3)
state.set('sensor.licznik_pradu_dol_kvar_py', value_kvar)
value_pf = round(int.from_bytes(value_decoded[8:10], byteorder='big')*0.001, 3)
state.set('sensor.licznik_pradu_dol_pf_py', value_pf)`
I did some Google; maybe this could help you further:
https://stackoverflow.com/questions/48885704/decode-base64-encoded-byte-array-to-negative-decimal-value-java-to-python
coded_string = "/bfbKNk="
bytes_val = base64.decodesbytes(coded_string.encode())
bval = "".join("{0:08b}".format(c) for c in bytes_val)
intval = int(bval, 2)
if bytes_val[0] & 0x70 != 0: # manually takes the two's complement
intval -= int('1' + '00' * len(bytes_val), 16)
print(intval)
Thank you for help. but I'm completlly new to python, where would I add this code?
but the value from the script is always positive
So, value is correct (except of sign), but positive? Or it is positive and incorrect?
The value is correct! But when the sun is out and my solar is working in the Tuya app I see the same value as in the py script but with a negative sign. When solar is not working everything is ok.
I think, it is one more bit/byte/DP especially for the direction. Need to find :-) I see you used the bytes [2:4],[6:8],[8:10],[11:13],[13:15] What is the values in bytes 0,1,4,5? It is possible the sign of direction there..
Ok, so I think it is the [0:1] bit! it gives me AA (0) or AQ (1) value and when checking the history on HA and on the tuya app I see that the value changed from AA to AQ when the solar started working and my power consumption from grid fell to 0. Now can you help me get this to the python code? I could check for a couple of days to see if it works ok.
@DominikWrobel Give me please the full raw (encoded, as received from device) data strings with positive and negative values.
@DominikWrobel Something like this:
value_power = round(int.from_bytes(value_decoded[2:4], byteorder='big')*0.001, 3) * (-1 if ord(value_decoded[0:1]) else 1)
@MaximumSU thank you - I think this will do the trick. I will check the resoults next couple of days and let you know if it works as it should.
Edit: 18.02.23 - everything works as it should the positive and negative values or spot on with the app.
@MaximumSU I am using your fork. How should I fill in the Binary Segmentation in your localtuya UI? It always my format is wrong. I should be using '0:2';'3:5';'6:8'. The only time I got it success is to just enter 0:2 and it gave me the voltage.
@mwkchan
I am using your fork. How should I fill in the Binary Segmentation in your localtuya UI? It always my format is wrong. I should be using '0:2';'3:5';'6:8'. The only time I got it success is to just enter 0:2 and it gave me the voltage.
- Just to confirm: are you use 3 different sensors? You not write '0:2';'3:5';'6:8' as segmentation rule in one sensor? You shoud have 3 different sensors...
- You shoud not use the positions in the two arguments, but the beginning and count of bytes. For example - not '3:5' but '3:2'. '3:5' means '5 bytes from position 3' that is wrong.
@MaximumSU Thank you for that. It's perfect now.
Are you going to ask the main repo in HACS incorporating your contributions? I think it is the way to go as seems more Tuya devices are returning states in such manner.
@mwkchan
Are you going to ask the main repo in HACS incorporating your contributions?
Yes, i did. It is here #1221 But it is not accepted yet.
@MaximumSU I'm using your fork but having some problems on one of the DP, this one:
{
"code": "lunar_cycle",
"custom_name": "",
"dp_id": 102,
"time": 1696591156422,
"value": "ARY="
},
It has 2 bytes that decoding gives this: 01 16
meaning 01 is on (oo was off) and the 2nd value converting 16 hex to decimal is 22 that is the lunar calendar day number.
But the real problem is that I add this DP and allways get this error:
and can't find any solution.
The ID is added like this (I need only the second byte for the moment):
Any tip or solution for this problem? Thanks.
@BaltasarParreira Is it any related errors in HASS log? Try to set the Scaling Factor to "1" Can you show the part of core.config_entries related to device (with local_key redacted)?
@MaximumSU You mean this (exert from core.config_entries file):
"bffad3aa56e6417a01mbsa": {
"friendly_name": "Aqua Reef Switch All",
"host": "192.168.0.150",
"local_key": "xxxxxxxxxxxxx",
"protocol_version": "3.3",
"enable_debug": true,
"manual_dps_strings": "101,102,103,106,107,108,109,110",
"reset_dpids": "",
"entities": [
{
"friendly_name": "Aqua Reef Switch All",
"restore_on_reconnect": false,
"is_passive_entity": false,
"id": 101,
"platform": "switch"
},
{
"friendly_name": "Aqua Reef Mode",
"select_options": "curve;lighting;fast;show",
"restore_on_reconnect": false,
"is_passive_entity": false,
"id": 103,
"platform": "select"
},
{
"friendly_name": "Aqua Reef Current Time",
"min_value": 0.0,
"max_value": 100000.0,
"step_size": 1.0,
"restore_on_reconnect": false,
"is_passive_entity": false,
"id": 110,
"platform": "number"
},
{
"friendly_name": "Aqua Reef Ch Data",
"base64_decode": true,
"bytes_range": "3:1",
"id": 108,
"platform": "sensor"
},
{
"friendly_name": "test",
"base64_decode": true,
"bytes_range": "1:1",
"id": 108,
"platform": "sensor"
},
{
"friendly_name": "Aqua Reef Lunar Cycle",
"base64_decode": true,
"bytes_range": "1:1",
"scaling": 1.0,
"id": 102,
"platform": "sensor"
}
],
"add_entities": false,
"device_id": "bffad3aa56e6417a01mbsa",
"dps_strings": [
"101 (value: True)",
"103 (value: curve)",
"110 (value: 6)",
"102 (value: -1)",
"106 (value: -1)",
"107 (value: -1)",
"108 (value: -1)",
"109 (value: -1)"
],
"product_key": "xxxxxxxxxxxxx"
}
The only thing I can see in the log regarding DP 102 is this:
2023-10-06 13:23:01.153 WARNING (SyncWorker_8) [custom_components.localtuya.sensor] [bff...bsa] Entity sensor.aqua_reef_lunar_cycle is requesting unknown DPS index 102
Changing the scaling doesn't make any difference for the 102, all the other are working 100% ok.
EDIT: Found this also on the main log page:
@MaximumSU yes, the tuya dev site shows it and the values changed when I change things for "Lunar cycle" in the Tuya app on the phone.
Here is the DPs checking on tuya website (Parameter(Request Method: GET)):
{
"result": {
"properties": [
{
"code": "switch_all",
"custom_name": "",
"dp_id": 101,
"time": 1696454949924,
"value": true
},
{
"code": "lunar_cycle",
"custom_name": "",
"dp_id": 102,
"time": 1696591156422,
"value": "ARY="
},
{
"code": "mode",
"custom_name": "",
"dp_id": 103,
"time": 1696590561024,
"value": "curve"
},
{
"code": "accalimation",
"custom_name": "",
"dp_id": 106,
"time": 1696454950122,
"value": "AAECZA=="
},
{
"code": "edit_curve",
"custom_name": "",
"dp_id": 107,
"time": 1696454950227,
"value": "FBAAADIAAAQAMgAABwAAAABAAAAAAEIACgAFRAAZAApGBS0FD0gPUBkUSh5aMiNMMmRkPFcyZGQ8WR5aMiNbFFAZFF0FMgUKXgAyAAVfADIAAA=="
},
{
"code": "Ch_data",
"custom_name": "",
"dp_id": 108,
"time": 1696597980708,
"value": "AAAAAA=="
},
{
"code": "ch_control",
"custom_name": "",
"dp_id": 109,
"time": 1693939181193
},
{
"code": "current_time",
"custom_name": "",
"dp_id": 110,
"time": 1696597201544,
"value": 56
}
]
},
"success": true,
"t": 1696597991048,
"tid": "1948aaf4644a11eebb8942dfc87a0fc7"
}
Also log shows a lot of this type of lines for a few different DPs, but none for 102:
2023-10-06 14:16:01.133 DEBUG (MainThread) [custom_components.localtuya.pytuya] [bff...bsa] Deciphered data = '{"dps":{"108":"AAAAAA=="},"t":1696598160}'
and
2023-10-06 14:15:57.684 DEBUG (MainThread) [custom_components.localtuya.pytuya] [bff...bsa] Deciphered data = '{"dps":{"101":true,"103":"curve","110":57}}'
@MaximumSU I just finish test all DPs that have raw value and the only one that works is the 108 that I already have 2 setup, all the others are not working and give the same problem as I reported for the 102. I mean 102, 106, 107 and 109 that I don't know what is for since doesn't have value and nothing changes when I change anything on the Tuya phone app.
@BaltasarParreira Are you tried "DPIDs to send in RESET command (separated by commas ',')- Used when device does not respond to status requests after turning on (optional)" in device settings?
@MaximumSU No, still no go. I put all DPs that are not working on the reset but none of those are working.
Should I put on the manual DPs the ones that are already auto detected or just the extra ones? As you can see on the image my setup is like that now.
@BaltasarParreira What is the name of your device? What is the HASS entity name of your device? Please enable debug for your device and restart Home Assistant. I need to see the startup log.
PS:Your issue is not related to my modifications (data from device just not come to HASS, there are may be many reasons for that), it is somethimg else. But i try to help you anyway :-)
@MaximumSU Yeahh is not your software as the any other tuya soft I get the same problem, but it's strange the 108 working ok and all the others not, anyway this is the main entity for my device (circle in red):
And the entity name on HASS is: switch.aqua_reef_switch_all
Regarding the debug log I will put those settings and reboot, but after where I can find it? In the normal system log?
Thanks for trying to help to solve all this.