[BUG] Tuya data processing - ZCL frame multiple attributes

[rforro]

Describe the bug Lidl Parkside water valve (under development), which is as usual a Tuya device publishes one ZCL frame with two attributes+values. This structure cannot be deserialized correctly using any of two Tuya Manufucturing Clusters. In one case the second one is completely ignored (a warning is issued, see log) in other case both attributes will get value from the second one.

Here is the complete ZCL Frame: b'\x09\x7B\x02\x01\x0F\x01\x01\x00\x01\x01\x05\x02\x00\x04\x00\x00\x00\x07' first attribute is onoff \x01\x01\x00\x01\x01, second attribute is timer duration \x05\x02\x00\x04\x00\x00\x00\x07.

I could narrow the deserializing problem to class TuyaData https://github.com/zigpy/zha-device-handlers/blob/b802c1fb2cf2682f9a4722bfb57a1958cad9dad7/zhaquirks/tuya/init.py#L205-L210 (I would like to stick with TuyaNewManufCluster)

In theory the TuyaData class could during deserializing check if there is some more data to parse, but how to run this process again? I have to idea how to solve it, do any of you can point me to right direction?

Complete log

2022-05-15 07:00:23 DEBUG (MainThread) [zigpy.zcl] [0x2F34:1:0xef00] Received ZCL frame: b'\x09\x7B\x02\x01\x0F\x01\x01\x00\x01\x01\x05\x02\x00\x04\x00\x00\x00\x07'
2022-05-15 07:00:23 DEBUG (MainThread) [zigpy.zcl] [0x2F34:1:0xef00] Decoded ZCL frame header: ZCLHeader(frame_control=FrameControl(frame_type=<FrameType.CLUSTER_COMMAND: 1>, is_manufacturer_specific=0, is_reply=1, disable_default_response=0, reserved=0, *is_cluster=True, *is_general=False), tsn=123, command_id=2, *is_reply=True)
2022-05-15 07:00:23 DEBUG (MainThread) [zigpy.zcl] [0x2F34:1:0xef00] Decoded ZCL frame: TuyaWaterValveManufCluster:set_data_response(data=TuyaCommand(status=1, tsn=15, dp=1, data=TuyaData(dp_type=<TuyaDPType.BOOL: 1>, function=0, raw=b'\x01', *payload=<Bool.true: 1>)))
2022-05-15 07:00:23 WARNING (MainThread) [zigpy.zcl] [0x2F34:1:0xef00] Data remains after deserializing ZCL frame: b'\x05\x02\x00\x04\x00\x00\x00\x07'
2022-05-15 07:00:23 DEBUG (MainThread) [zigpy.zcl] [0x2F34:1:0xef00] Received command 0x02 (TSN 123): set_data_response(data=TuyaCommand(status=1, tsn=15, dp=1, data=TuyaData(dp_type=<TuyaDPType.BOOL: 1>, function=0, raw=b'\x01', *payload=<Bool.true: 1>)))

[MattWestb]

Z2M was adding the multiple command sent in one frame in one update in the beginning of the year so i think its coming more MCU that need this functionality in the near future.

Also then cooking quirks its needed or can missing command received from the device and cant see that functions need being added for some DPS.

[rforro]

Shortcut to how z2m solved it https://github.com/Koenkk/zigbee-herdsman/pull/483

[puddly]

If I'm understanding the problem correctly, I think this might work (feel free to rename the struct, I'm not too familiar with the Tuya protocol):

diff --git a/zhaquirks/tuya/__init__.py b/zhaquirks/tuya/__init__.py
index 1164358..14679da 100644
--- a/zhaquirks/tuya/__init__.py
+++ b/zhaquirks/tuya/__init__.py
@@ -262,13 +262,18 @@ class Data(t.List, item_type=t.uint8_t):
         return value
 
 
+class TuyaDatapointAndData(t.Struct):
+    dp: t.uint8_t
+    data: TuyaData
+
+
 class TuyaCommand(t.Struct):
     """Tuya manufacturer cluster command."""
 
     status: t.uint8_t
     tsn: t.uint8_t
-    dp: t.uint8_t
-    data: TuyaData
+    datapoints: t.List[TuyaDatapointAndData]
 
 
 class TuyaManufCluster(CustomCluster):

Example:

In [1]: from unittest.mock import Mock

In [2]: from zhaquirks.tuya.mcu import TuyaNewManufCluster

In [3]: ep = Mock()  # fake endpoint object

In [4]: data = b'\x09\x7B\x02\x01\x0F\x01\x01\x00\x01\x01\x05\x02\x00\x04\x00\x00\x00\x07'

In [5]: TuyaNewManufCluster(ep).deserialize(data)
Out[5]:
(ZCLHeader(frame_control=FrameControl(frame_type=<FrameType.CLUSTER_COMMAND: 1>, is_manufacturer_specific=0, is_reply=True, disable_default_response=0, reserved=0, *is_cluster=True, *is_general=False), tsn=123, command_id=2, *is_reply=True),
 set_data_response(data=TuyaCommand(status=1, tsn=15, datapoints=[TuyaDatapointAndData(dp=1, data=TuyaData(dp_type=<TuyaDPType.BOOL: 1>, function=0, raw=b'\x01', *payload=<Bool.true: 1>)), TuyaDatapointAndData(dp=5, data=TuyaData(dp_type=<TuyaDPType.VALUE: 2>, function=0, raw=b'\x07\x00\x00\x00', *payload=7))])))

Unit tests will need to be updated for this to work.

[MattWestb]

Was finding some interesting tuya dev docks that is showing how multiple DP is being sent: https://developer.tuya.com/en/docs/iot/tuya-zigbee-module-uart-communication-protocol?id=K9ear5khsqoty#title-14-Send%20commands.

[javicalle]

I believe that #1779 should have fixed this issue.

[dmulcahey]

fixed in #1779