Doubled Reporting Line for the same Attribute / Interval values do not change properly

[MMi1603]

What happened?

Since the change from Zigbee2MQTT 1.39.1 to 1.40 no Interval in the Reporting Line can be changed properly. Two lines with the same Attribute appear, but with interval values (prev and new, same value or even the same). Deleting one of them always result back in the old value. Keeping both lines seem to work with the new value.

Going back to a 1.39.1 backup did not show the problem. Upgrading again to 1.40.1 also had the issue.

I saw this for different vendors (Ikea, Tuya, Philips) on different attributes (remaining battery, voltage or power reporting interval, both battery devices and controllers).

Looking forward to your kind responses, also to find out if this is an issue on my setup only or if it occurs also for other users.

What did you expect to happen?

Clear attribute value. No multiple lines for each attribute.

How to reproduce it (minimal and precise)

1. Update to 1.40.0 or 1.40.1
2. Change any interval value and apply.

Zigbee2MQTT version

1.40.1

Adapter firmware version

20240710

Adapter

SONOFF Zigbee 3.0 USB Dongle Plus

Setup

Z2M to MQTT to Openhab on Raspberry OS

Debug log

log.log

[MMi1603]

I could realize in 1.40.1 that after removing and re-pairing a device or adding a complete new device everything is normal again and attributes can be changed as usual on the new added device. The unchanged devices I had already in 1.39 still have the described problem. So the error must have been by the change from 1.39 to 1.40/1.40.1.

I would like to avoid re-pairing all devices again, so any suggestion would be welcome. Did this behaviour really happened on my setup only or did anybody else have the same problem?

Regards, MMi1603

[MMi1603]

I have tried the following steps:

• un- and reinstall node and node js (both 20.7.0 now)
• un- and reinstall zigbee2mqtt with using a Z2M backup in /opt/zigbee2mqtt/data
• upgrade Zigbee to 1.40.2 with successful upgrade herdsman to 2.1.3 and herdsman converter to 20.21.0

Unfortunately still with the same result: When I add a new device under 1.40 (0 or 1) I can change attributes properly. For all other devices which were already paired in 1.39 or earlier I cannot change attributes properly.

Two lines appear first with the same content, after leaving the page also with different contents (see picture with an Ikea Styrbar and 1.40.2 as example). After disabling any of the two lines always the line with the old status remain, which also cannot be removed totally then.

When using a full Raspberry backup with zigbee2mqtt 1.39 on it everything is fine, after upgrading Z2M to 1.40.x the same problem appears again.

A helping comment would be welcome, Regards, MMi1603

[iambucketdotcom]

I too am having this same issue, and can repeat steps until im blue in the face with the same result, older devices will not properly report, throwing my energy monitoring off... Going into reporting and pressing "Apply" on each of the entries creates a duplicate entry, and only then will things work...

Remove any of them, and it all stops reporting normally... if I remove the device and re-add its fine... but with nearly 50 Zigbee devices, I don't want to remove and re-pair all of them.

[fuzzy01]

This issue is real. Old reporting entries cannot be deleted or modified only new ones. This due to a change in the content of the configuredReportings entry in database.db. The manufacturerCode is not written in the database file and the code does not handle the old entries correctly. I compared the database entries for an Aquara plug.

Original pre 1.40

            "configuredReportings": [
                {
                    "cluster": 6,
                    "attrId": 0,
                    "minRepIntval": 0,
                    "maxRepIntval": 3600,
                    "repChange": 0,
                    "manufacturerCode": null
                },
                {
                    "cluster": 1794,
                    "attrId": 0,
                    "minRepIntval": 5,
                    "maxRepIntval": 3600,
                    "repChange": 0,
                    "manufacturerCode": null
                }
             ],

After a reconfigure on the GUI in 1.40.2 entries are duplicated

            "configuredReportings": [
                {
                    "cluster": 6,
                    "attrId": 0,
                    "minRepIntval": 0,
                    "maxRepIntval": 3600,
                    "repChange": 0,
                    "manufacturerCode": null
                },
                {
                    "cluster": 1794,
                    "attrId": 0,
                    "minRepIntval": 5,
                    "maxRepIntval": 3600,
                    "repChange": 0,
                    "manufacturerCode": null
                },
                {
                    "cluster": 6,
                    "attrId": 0,
                    "minRepIntval": 0,
                    "maxRepIntval": 3600,
                    "repChange": 0
                },
                {
                    "cluster": 1794,
                    "attrId": 0,
                    "minRepIntval": 5,
                    "maxRepIntval": 3600,
                    "repChange": 0
                }
            ],

After repairing the plug the duplicate lines disappear:

            "configuredReportings": [
                {
                    "cluster": 6,
                    "attrId": 0,
                    "minRepIntval": 0,
                    "maxRepIntval": 3600,
                    "repChange": 0
                },
                {
                    "cluster": 1794,
                    "attrId": 0,
                    "minRepIntval": 5,
                    "maxRepIntval": 3600,
                    "repChange": 0
                }
            ],

[fuzzy01]

@Koenkk please fix this. This issue leads to duplicate bogus entries in database.db If was fixed in https://github.com/nurikk/zigbee2mqtt-frontend/issues/1829 but returned with 1.40 I tested with 1.41 and the bug still exists

[fuzzy01]

Until it gets fixed here is how you can fix database.db

1. Stop zigbee2mqtt
2. cd to directory where database.db is located
3. enter these commands

cp database.db database.db.bak
sed -E 's/,"manufacturerCode":null//g' database.db.bak > database.db

4. Start zigbee2mqtt

Duplicated reporting entries must be deleted on the gui manually.

[MMi1603]

Until it gets fixed here is how you can fix database.db

1. Stop zigbee2mqtt
2. cd to directory where database.db is located
3. enter these commands

cp database.db database.db.bak
sed -E 's/,"manufacturerCode":null//g' database.db.bak > database.db

4. Start zigbee2mqtt

Duplicated reporting entries must be deleted on the gui manually.

Thanks for your support and your proposal. I had started to re-pair my battery devices, but with this (and a future fix) I can avoid further ones.

[MMi1603]

With fuzzy01's proposal on 1.40.1 and a later update to 1.41 the problem was solved. I hope, this works also for all others. I guess I can close this issue now. Thanks again, MMi1603

[jamesonuk]

@Koenkk I came to report the same but I am wondering whether this is purely cosmetic or whether this might cause issues?

This must be stored on the device itself as you get the timeout error when trying to set if the device isn't awake but if there are two entries shown in the UI and I disable one, could this end up with the UI looking right and showing one entry but the device having removed the actual reporting ?

[Koenkk]

@jamesonuk this is purely a cosmetic issue

[Wielebny666]

But it still exists

[emandtf]

It continue to happen....... it's quite frustrating.....

[ben33880]

Same here, problem with z2m 2.0

[HermesHonshappo]

+1, running the HA addon of Z2M in its latest version 2.0.0-2, and still happens

[basrieter]

Same problem here.

[fuzzy01]

I've already posted a manual fix for this issue. However, because the original issuer closed this ticket, please open a new one if you need further assistance.