zigbee2mqtt problem wih ember driver and joining develco devices

What happened?

can'y join frient heat alarm with ember driver. Tried directly on coordinator and multiple routers.

What did you expect to happen?

joining works with ezsp driver

How to reproduce it (minimal and precise)

No response

Zigbee2MQTT version

1.37.0-dev commit: 358e2ee

Adapter firmware version

7.4.3.0 build 0

Adapter

Ezsp or ember (sonoff plus e)

Setup

z2n in docker

Debug log

[2024-05-06 07:15:25] error: z2m: Failed to configure 'Kitchen Smoke Sensor', attempt 1 (Error: Bind 0x0015bc00340008dd/35 ssIasZone from '0xe0798dfffe7d7366/1' failed ([ZDO] Failed response by NCP for "42235" cluster "32801" with status=ZDP_TABLE_FULL.) at EmberOneWaitress.resolveZDO (/app/node_modules/zigbee-herdsman/src/adapter/ember/adapter/oneWaitress.ts:138:35) at EmberAdapter.onZDOResponse (/app/node_modules/zigbee-herdsman/src/adapter/ember/adapter/emberAdapter.ts:573:26) at Ezsp.emit (node:events:517:28) at Ezsp.ezspIncomingMessageHandler (/app/node_modules/zigbee-herdsman/src/adapter/ember/ezsp/ezsp.ts:4524:22) at Ezsp.callbackDispatch (/app/node_modules/zigbee-herdsman/src/adapter/ember/ezsp/ezsp.ts:823:18) at Ezsp.tick (/app/node_modules/zigbee-herdsman/src/adapter/ember/ezsp/ezsp.ts:449:22) at listOnTimeout (node:internal/timers:569:17) at processTimers (node:internal/timers:512:7))

[2024-05-06 07:20:28] error: z2m: Failed to configure 'Kitchen Smoke Sensor', attempt 3 (Error: Bind 0x0015bc00340008dd/35 ssIasZone from '0xe0798dfffe7d7366/1' failed ([ZDO] Failed response by NCP for "42235" cluster "32801" with status=ZDP_TABLE_FULL.) at EmberOneWaitress.resolveZDO (/app/node_modules/zigbee-herdsman/src/adapter/ember/adapter/oneWaitress.ts:138:35) at EmberAdapter.onZDOResponse (/app/node_modules/zigbee-herdsman/src/adapter/ember/adapter/emberAdapter.ts:573:26) at Ezsp.emit (node:events:517:28) at Ezsp.ezspIncomingMessageHandler (/app/node_modules/zigbee-herdsman/src/adapter/ember/ezsp/ezsp.ts:4524:22) at Ezsp.callbackDispatch (/app/node_modules/zigbee-herdsman/src/adapter/ember/ezsp/ezsp.ts:823:18) at Ezsp.tick (/app/node_modules/zigbee-herdsman/src/adapter/ember/ezsp/ezsp.ts:449:22) at listOnTimeout (node:internal/timers:569:17) at processTimers (node:internal/timers:512:7)) [2024-05-06 18:02:57] error: zh:controller: Interview failed for '0x0015bc00340008dd with error 'Error: {"target":42235,"apsFrame":{"profileId":0,"clusterId":4,"sourceEndpoint":0,"destinationEndpoint":0,"options":4416,"groupId":0,"sequence":160},"responseClusterId":32772} timed out after 15000ms' [2024-05-06 18:02:57] error: z2m: Failed to interview '0x0015bc00340008dd', device has not successfully been paired [2024-05-06 18:03:44] error: z2m: Failed to configure '0x0015bc00340008dd', attempt 4 (Error: Bind 0x0015bc00340008dd/35 genPowerCfg from '0xe0798dfffe7d7366/1' failed (Delivery failed for {"profileId":0,"clusterId":33,"sourceEndpoint":0,"destinationEndpoint":0,"options":4416,"groupId":0,"sequence":178}) at EmberOneWaitress.deliveryFailedFor (/app/node_modules/zigbee-herdsman/src/adapter/ember/adapter/oneWaitress.ts:96:31) at EmberAdapter.onMessageSentDeliveryFailed (/app/node_modules/zigbee-herdsman/src/adapter/ember/adapter/emberAdapter.ts:558:30) at Ezsp.emit (node:events:517:28) at Ezsp.ezspMessageSentHandler (/app/node_modules/zigbee-herdsman/src/adapter/ember/ezsp/ezsp.ts:3958:18) at Ezsp.callbackDispatch (/app/node_modules/zigbee-herdsman/src/adapter/ember/ezsp/ezsp.ts:795:18) at Ezsp.tick (/app/node_modules/zigbee-herdsman/src/adapter/ember/ezsp/ezsp.ts:449:22) at listOnTimeout (node:internal/timers:569:17) at processTimers (node:internal/timers:512:7)) [2024-05-06 18:03:47] warning: zh:ember:ezsp: Received network/route error ROUTE_ERROR_INDIRECT_TRANSACTION_EXPIRY for "42235". [2024-05-06 18:03:49] warning: zh:ember:ezsp: Received network/route error ROUTE_ERROR_INDIRECT_TRANSACTION_EXPIRY for "42235". [2024-05-06 18:03:50] warning: zh:ember:ezsp: Received network/route error ROUTE_ERROR_INDIRECT_TRANSACTION_EXPIRY for "42235". [2024-05-06 18:04:19] warning: zh:ember: [TRUST CENTER] Device 42235:0x0015bc00340008dd was denied joining via 40337. [2024-05-06 18:04:19] warning: zh:ember: [TRUST CENTER] Device 42235:0x0015bc00340008dd was denied joining via 40337. [2024-05-06 18:04:19] warning: z2m: Device '0x0015bc00340008dd' left the network [2024-05-06 18:04:42] warning: zh:ember: [TRUST CENTER] Device 29742:0x0015bc00340008dd was denied joining via 40337. [2024-05-06 18:04:42] warning: zh:ember: [TRUST CENTER] Device 29742:0x0015bc00340008dd was denied joining via 40337. [2024-05-06 18:05:20] error: zh:controller: Interview failed for '0x0015bc00340008dd with error 'Error: [ZDO] Failed response by NCP for "61782" cluster "32772" with status=ZDP_INVALID_REQUEST_TYPE.' [2024-05-06 18:05:20] error: z2m: Failed to interview '0x0015bc00340008dd', device has not successfully been paired [2024-05-06 18:13:47] warning: zh:ember:ezsp: Received network/route error ROUTE_ERROR_NO_ROUTE_AVAILABLE for "54868". [2024-05-06 18:16:40] warning: z2m: Device 'Kitchen Smoke Sensor' left the network [2024-05-06 18:17:04] warning: zh:ember: [TRUST CENTER] Device 33160:0x0015bc00340008dd was denied joining via 60665. [2024-05-06 18:17:05] warning: zh:ember: [TRUST CENTER] Device 33160:0x0015bc00340008dd was denied joining via 60665. [2024-05-06 18:17:28] warning: zh:ember: [TRUST CENTER] Device 28037:0x0015bc00340008dd was denied joining via 5159. [2024-05-06 18:17:28] warning: zh:ember: [TRUST CENTER] Device 28037:0x0015bc00340008dd was denied joining via 5159. [2024-05-06 18:17:51] warning: zh:ember: [TRUST CENTER] Device 48604:0x0015bc00340008dd was denied joining via 5159. [2024-05-06 18:17:51] warning: zh:ember: [TRUST CENTER] Device 48604:0x0015bc00340008dd was denied joining via 5159. [2024-05-06 18:18:14] warning: zh:ember: [TRUST CENTER] Device 61138:0x0015bc00340008dd was denied joining via 5159. [2024-05-06 18:18:14] warning: zh:ember: [TRUST CENTER] Device 61138:0x0015bc00340008dd was denied joining via 5159. [2024-05-06 18:19:15] warning: zh:ember:ezsp: Received network/route error ROUTE_ERROR_SOURCE_ROUTE_FAILURE for "50845". [2024-05-06 18:19:15] warning: zh:ember:ezsp: Received network/route error ROUTE_ERROR_MANY_TO_ONE_ROUTE_FAILURE for "50845". [2024-05-06 18:21:17] error: zh:controller: Interview failed for '0x0015bc00340008dd with error 'Error: [ZDO] Failed response by NCP for "26597" cluster "32772" with status=ZDP_INVALID_REQUEST_TYPE.' [2024-05-06 18:21:17] error: z2m: Failed to interview 'Kitchen Smoke Sensor', device has not successfully been paired

May 06 '24 15:05 mamrai1

joining is ok with ezsp driver.

May 06 '24 15:05 mamrai1

Also after joining the smoke detectors with ezsp and then using ember there is battery drainage issue on device connected directly on coordinator.

May 07 '24 12:05 mamrai1

Can you provide a log for ezsp that's roughly the same actions as the ones in the ember log above? Your device is replying ZDP_TABLE_FULL at first, which means it can't bind anymore stuff. Did you try to do a full reset of the device (see if there is a procedure for it)?

May 07 '24 15:05 Nerivec

Yes I will reset one device tonight and send the logs.

May 07 '24 15:05 mamrai1

ember.log ezsp.log attached 2 debug logs when resetting and joining office smoke sensor(develco) with ezsp and ember

May 07 '24 18:05 mamrai1

The denied join issue should be fixed after this is merged/available https://github.com/Koenkk/zigbee-herdsman/pull/1052 Still looking into the ZDO issue.

May 07 '24 22:05 Nerivec

Tried to join to the network today: Remote Control IKEA E1524/E1810 and Aeotec Range extender WG001. Many attempts without success, after changing in config ember to ezsp, devices joined in first try. Back to ember, devices works.

May 09 '24 15:05 roon7

Had the same issue with adding HUE bulbs. Switched to Z2M Edge version and it helped.

May 09 '24 17:05 CrazyCoder

@mamrai1 Can you confirm if that device is still sending back ZDO errors with 1.37.1?

May 10 '24 00:05 Nerivec

No stil can't join the sensor. logs3 with latest commit. ezsp3.log ember3.log ezsp2.log ember2.log

May 10 '24 07:05 mamrai1

I had the same issue with a Gynoid (Orvibo OEM) brand motion sensor, and after updating Z2M to version 1.37.1 and re-pairing the sensor, the problem was resolved.

May 11 '24 00:05 juanchula

@mamrai1 do you by any chance have hardware to sniff while joining this device? https://www.zigbee2mqtt.io/advanced/zigbee/04_sniff_zigbee_traffic.html . If yes, could you make a sniff of ember (should fail) and ezsp (should work)

May 14 '24 18:05 Koenkk

@mamrai1 do you by any chance have hardware to sniff while joining this device? https://www.zigbee2mqtt.io/advanced/zigbee/04_sniff_zigbee_traffic.html . If yes, could you make a sniff of ember (should fail) and ezsp (should work)

ezsp_ember.zip

Jun 01 '24 21:06 mamrai1

The joins fails due to the simple descriptor request failing. This is an issue on the device side, the same happens on ezsp but the difference is that the device immediately joins again after which the simple descriptor request succeeds:

EZSP:

Ember:

With ember, could you try to re-pair the device immediately after interviewing fails?

Jun 02 '24 19:06 Koenkk

ember.zip after hitting the button on the webgui it succeeds at interview but fails later at configuring the sensor @Koenkk

Jun 02 '24 20:06 mamrai1

@Nerivec @Koenkk any feedback ?

Jun 23 '24 15:06 mamrai1

Could you try to configure it again via the frontend? It's the yellow configure button. Before doing that make sure to wakeup the device right before clicking the button.

Jun 24 '24 08:06 Koenkk

No, it fails with ember driver. (status=ZDP_TABLE_FULL. ) At least after joining the device with ezsp and changing to ember it works..Also the interview button works with no errors.

Jun 24 '24 09:06 mamrai1

2024-06-24 12:40:53z2m: Failed to configure 'Office Smoke Sensor', attempt 1 (Error: Bind 0x0015bc00340009e6/35 genBasic from '0xe0798dfffe7d7366/1' failed ([ZDO] Failed response by NCP for "16827" cluster "32801" with status=ZDP_TABLE_FULL.) at EmberOneWaitress.resolveZDO (/app/node_modules/zigbee-herdsman/src/adapter/ember/adapter/oneWaitress.ts:138:35) at EmberAdapter.onZDOResponse (/app/node_modules/zigbee-herdsman/src/adapter/ember/adapter/emberAdapter.ts:663:26) at Ezsp.emit (node:events:517:28) at Ezsp.ezspIncomingMessageHandler (/app/node_modules/zigbee-herdsman/src/adapter/ember/ezsp/ezsp.ts:4517:22) at Ezsp.callbackDispatch (/app/node_modules/zigbee-herdsman/src/adapter/ember/ezsp/ezsp.ts:820:18) at Ezsp.tick (/app/node_modules/zigbee-herdsman/src/adapter/ember/ezsp/ezsp.ts:446:22) at listOnTimeout (node:internal/timers:569:17) at processTimers (node:internal/timers:512:7))

At least the situation is better now. Switching to ember and intervewing the device works (blue button) with no errors. Also changing reporting values works as expected.. . Zigbee2MQTT version: 1.38.0-dev commit: ee42d68 EmberZNet Coordinator revision 7.4.3 [GA]

Jun 24 '24 09:06 mamrai1

Even though it sometimes works, I suspect this is more about luck, since the driver has zero control over how the device responds to a command being sent to it (ZDP_TABLE_FULL is reported directly from the device, means the binding table of that device is full, the only thing the z2m driver does is read said status when it receives it). Combining your previous logs, errors are also not consistent, since it failed at different steps during interviews with different errors. Looks like a firmware mishap of some kind on that device.

Jun 24 '24 17:06 Nerivec

Relating to ZDP_TABLE_FULL: I went over the bind part of the code. It seems other drivers (ezsp, zstack) are not checking the status (ZDO) of the bind response. Leaves me thinking either:

That develco device is not reporting the proper status even if successful, so it works if you just ignore it
The bind always fails on that develco device but other drivers let Z2M think it didn't

I glanced over the rest of the ZDO requests, seems the same applies for a few of them (most have a status in response). @Koenkk can you confirm this for zstack (at least the part about the status not being checked against)?

Jul 17 '24 21:07 Nerivec

@Nerivec zstack indeed doesn't check the return code.

@mamrai1

Could you check if the issue is fixed with the following external converter:

save this as file next to configuration.yaml as ext_converter.js
add it to configuration.yaml:

external_converters:
  - ext_converter.js

start z2m, check if issue is fixed

Jul 18 '24 18:07 Koenkk

2024-07-18 22:06:55] error: z2m: Failed to load external converter file 'ext_converter.js' (Cannot find module '../lib/develco' Require stack:

/app/dist/util/utils.js
/app/dist/util/settings.js
/app/index.js)

Jul 18 '24 19:07 mamrai1

Change a couple of the top lines, from:

const {develcoModernExtend} = require('../lib/develco');
const {battery} = require('../lib/modernExtend');

to:

const {develcoModernExtend} = require('zigbee-herdsman-converters/lib/develco');
const {battery} = require('zigbee-herdsman-converters/lib/modernExtend');

Jul 18 '24 19:07 Nerivec

Warning 2024-07-19 00:17:55zh:ember: [ZDO] Node descriptor for '11734' reports device is only compliant to revision '21' of the ZigBee specification (current revision: 22). Error 2024-07-19 00:18:00z2m: Request 'zigbee2mqtt/bridge/request/device/interview' failed with error: 'interview of 'Kitchen Smoke Sensor' (0x0015bc00340008dd) failed: Error: Delivery failed for {"profileId":0,"clusterId":4,"sourceEndpoint":0,"destinationEndpoint":0,"options":4416,"groupId":0,"sequence":223}' Error 2024-07-19 00:18:23z2m: Failed to configure 'Kitchen Smoke Sensor', attempt 1 (ReferenceError: logger is not defined at configure (/app/data/extension/externally-loaded.js:42:17) at runNextTicks (node:internal/process/task_queues:60:5) at processImmediate (node:internal/timers:447:9) at Object.configure (/app/node_modules/zigbee-herdsman-converters/src/index.ts:190:21) at Configure.configure (/app/lib/extension/configure.ts:126:13) at Configure.onMQTTMessage

Jul 18 '24 21:07 mamrai1

Argh, missing an import. Went over the entire thing, made a few more tweaks. Try this:

const fz = require('zigbee-herdsman-converters/converters/fromZigbee');
const tz = require('zigbee-herdsman-converters/converters/toZigbee');
const exposes = require('zigbee-herdsman-converters/lib/exposes');
const reporting = require('zigbee-herdsman-converters/lib/reporting');
const e = exposes.presets;
const ea = exposes.access;
const {develcoModernExtend} = require('zigbee-herdsman-converters/lib/develco');
const {battery} = require('zigbee-herdsman-converters/lib/modernExtend');
const {logger} = require('zigbee-herdsman-converters/lib/logger');

const definition =  {
    zigbeeModel: ['HESZB-120'],
    model: 'HESZB-120',
    vendor: 'Develco',
    description: 'Fire detector with siren',
    whiteLabel: [{vendor: 'Frient', model: '94431', description: 'Smart Intelligent Heat Alarm'}],
    fromZigbee: [fz.ias_smoke_alarm_1_develco, fz.ignore_basic_report, fz.ias_enroll, fz.ias_wd], //, develco.fz.fault_status],
    toZigbee: [tz.warning, tz.ias_max_duration, tz.warning_simple],
    // ota: ota.zigbeeOTA,
    extend: [
        develcoModernExtend.addCustomClusterManuSpecificDevelcoGenBasic(),
        develcoModernExtend.readGenBasicPrimaryVersions(),
        develcoModernExtend.temperature(), // TODO: ep 38
        battery({
            voltageToPercentage: '3V_2500',
            percentage: true,
            voltage: true,
            lowStatus: false,
            voltageReporting: true,
            percentageReporting: false,
        }),
    ],
    configure: async (device, coordinatorEndpoint) => {
        const endpoint = device.getEndpoint(35);

        for (const cluster of ['ssIasZone', 'ssIasWd', 'genBasic', 'genBinaryInput']) {
            try {
                await reporting.bind(endpoint, coordinatorEndpoint, [cluster]);
            } catch (error) {
                logger.debug(`Failed to bind '${cluster}' ${error}`, 'extconv:develco:heszb120');
            }
        }

        await endpoint.read('ssIasZone', ['iasCieAddr', 'zoneState', 'zoneId']);
        await endpoint.read('genBinaryInput', ['reliability', 'statusFlags']);
        await endpoint.read('ssIasWd', ['maxDuration']);
    },
    endpoint: (device) => {
        return {default: 35};
    },
    exposes: [
        e.smoke(),
        e.battery_low(),
        e.test(),
        e.numeric('max_duration', ea.ALL).withUnit('s').withValueMin(0).withValueMax(600).withDescription('Duration of Siren'),
        e.binary('alarm', ea.SET, 'START', 'OFF').withDescription('Manual Start of Siren'),
        e
            .enum('reliability', ea.STATE, ['no_fault_detected', 'unreliable_other', 'process_error'])
            .withDescription('Indicates reason if any fault'),
        e.binary('fault', ea.STATE, true, false).withDescription('Indicates whether the device are in fault state'),
    ],
};

module.exports = definition;

Jul 19 '24 01:07 Nerivec

Seems now that reconfiguring the device works! Thanks

Jul 19 '24 03:07 mamrai1

@Nerivec how do you proceed now? Fixing the converter for the Develco heat alarm or a more generic solution ignoring the ZDO response?

Jul 19 '24 13:07 mamrai1

We shouldn't ignore the ZDO response, since it is supposed to tell us if the bind was successful or not. Z2M should not assume it worked since that can quickly cascade into other issues.

As for this device, now we need to figure out which scenario is effective here, either the bind worked, but the device reports a wrong status (in that case, we can override the converter to bypass it), or the bind just always fails (in that case, remains to see how "working" the device really is).

Jul 19 '24 14:07 Nerivec

I've integrated the fixes into the Develco converter
I've created a fix to check for the status code with ZStack adapters (https://github.com/Koenkk/zigbee-herdsman/pull/1125)

I guess this can be closed now.

Jul 21 '24 10:07 Koenkk

zigbee2mqtt zigbee2mqtt copied to clipboard

problem wih ember driver and joining develco devices

What happened?

What did you expect to happen?

How to reproduce it (minimal and precise)

Zigbee2MQTT version

Adapter firmware version

Adapter

Setup

Debug log

zigbee2mqtt
zigbee2mqtt copied to clipboard