Report only when something changes

[vk496]

Hello!

I've just started using zlinky with Zigbee2mqtt (currently in development by @ralmn) and I noticed that zlinky report data massively (20 times per second?), which completely blocks the whole IO of the storage and the system (and increase the DB some GB every hour)

More precisely, the linkquality is the attribute which constantly changes. The rest of attributes retain the same for much more time.

My current workaround is disabling most of the entities , but it would be much more gentile if zlinky would report only when a attribute change.

Can someone confirm that this behaviour is not only in my side?

Br, Valentin

[djsyl]

Hello, in Z2M you have in settings (spécific) parameter measurement_poll_interval you have to restart z2m after change this and a last version 1.22.0 commit: b3e9afaf

have fun

[vk496]

Hello,

Thank you for your response @djsyl

Currently I'm limited to the previous version of Z2M (arm64), so I manually force a poll interval of 120 in the code.

However, this affect only non reportable attributes. The reportable attributes are sent over Zigbee network multiples times per second (firmware behaviour), which are forwarded to the MQTT network.

Moreover, I think Zlinky send ALL data each time, so I suspecting that all attributes are transmitted over Zigbee constantly + the 60s poll interval (which is irrelevant if I'm right).

I couldn't check it directly within the Zigbee network, but my home assistant DB was 200 MB after months of usage. After setting up the zlinky with only 2 attributes enabled (energy and PTEC), it increased 400 MB this night (with all attributes, consumes 1G in 2h)

BR, Valentin

[djsyl]

@vk496 vincent Je sais pas ce que vous avez fait mais j'ai aussi modifier mon fichier.js et mis des paramètres toutes les 15 secondes sur le pool sur une version 1.22 de zigbee2mqtt constatez dans le fichier gif ci-joint que l'on n'est loin de la saturation.

//lixilight2.js
const fz = require('zigbee-herdsman-converters/converters/fromZigbee');
const tz = require('zigbee-herdsman-converters/converters/toZigbee');
const exposes = require('zigbee-herdsman-converters/lib/exposes');
const globalStore = require('zigbee-herdsman-converters/lib/store');
const reporting = require('zigbee-herdsman-converters/lib/reporting');
const extend = require('zigbee-herdsman-converters/lib/extend');
const { repInterval } = require('zigbee-herdsman-converters/lib/constants');

const { precisionRound, postfixWithEndpointName } = require('zigbee-herdsman-converters/lib/utils');

const e = exposes.presets;
const ea = exposes.access;


// Options 

let option_standard = () => exposes.binary(`standard`, ea.SET, true, false).withDescription(`Compteur avec la TIC en mode standard`)
let option_triphase = () => exposes.binary(`triphase`, ea.SET, true, false).withDescription(`Compteur triphase`)
let option_production = () => exposes.binary(`production`, ea.SET, true, false).withDescription(`Compteur avec de la production`)



// All attributes avaialables
let data = [
    { "name": " BASE", "unit": "Wh", "clusterId": 1794, "attributId": 0, "comment": "", "description": "Index Base", "cluster": "seMetering", "attribut": "currentSummDelivered", "triphase": false, "production": false, "standard": false },
    { "name": "EAIT", "unit": "Wh", "clusterId": 1794, "attributId": 1, "comment": "Production", "description": "Energie active injectée totale (Triphasé)", "cluster": "seMetering", "attribut": "currentSummReceived", "triphase": true, "production": false, "standard": true },
    { "name": "PMAX", "unit": "W", "clusterId": 2820, "attributId": 1293, "comment": "Triphasé", "description": "Puissance maximale triphasée atteinte (Triphasé)", "cluster": "haElectricalMeasurement", "attribut": "activePowerMax", "triphase": true, "production": false, "standard": false },
    { "name": "PAPP", "unit": "VA", "clusterId": 2820, "attributId": 1295, "comment": "Triphasé", "description": "Puissance apparente (Triphasé)", "cluster": "haElectricalMeasurement", "attribut": "apparentPower", "triphase": true, "production": false, "standard": false },
    { "name": " HCHC", "unit": "Wh", "clusterId": 1794, "attributId": 256, "comment": "", "description": "Index HCHC", "cluster": "seMetering", "attribut": "currentTier1SummDelivered", "triphase": false, "production": false, "standard": false },
    { "name": " HCHP", "unit": "Wh", "clusterId": 1794, "attributId": 258, "comment": "", "description": "Index HCHP", "cluster": "seMetering", "attribut": "currentTier2SummDelivered", "triphase": false, "production": false, "standard": false },
    { "name": "EASF02", "unit": "Wh", "clusterId": 1794, "attributId": 258, "comment": "", "description": "Energie active soutirée Fournisseur, index 02", "cluster": "seMetering", "attribut": "currentTier2SummDelivered", "triphase": false, "production": false, "standard": true },
    { "name": "CCASN", "unit": "W", "clusterId": 2820, "attributId": 1291, "comment": "", "description": "Point n de la courbe de charge active soutirée", "cluster": "haElectricalMeasurement", "attribut": "activePower", "triphase": false, "production": false, "standard": true },
    { "name": "CCASN-1", "unit": "W", "clusterId": 2820, "attributId": 2315, "comment": "", "description": "Point n-1 de la courbe de charge active soutirée", "cluster": "haElectricalMeasurement", "attribut": "activePowerPhB", "triphase": false, "production": false, "standard": true },
    { "name": "UMOY1", "unit": "V", "clusterId": 2820, "attributId": 1297, "comment": "mono / triphasé", "description": "Tension moy. ph. 1", "cluster": "haElectricalMeasurement", "attribut": "averageRmsVoltageMeasPeriod", "triphase": false, "production": false, "standard": true },
    { "name": "IRMS1", "unit": "A", "clusterId": 2820, "attributId": 1288, "comment": "mono / triphasé", "description": "Courant efficace, phase 1", "cluster": "haElectricalMeasurement", "attribut": "rmsCurrent", "triphase": false, "production": false, "standard": true },
];

function createChunk(array, chunkSize) {
    let R = [];
    for (let i = 0; i < array.length; i += chunkSize) {
        R.push(array.slice(i, i + chunkSize));
    }
    return R;
}

function onlyUnique(value, index, self) {
    return self.indexOf(value) === index;
}

function extractAttribute(cluster, standard, triphase, production) {
    let attributes = data.filter(item => item.cluster === cluster);
    if (standard == false) {
        attributes = attributes.filter(a => a.standard == false);
    }
    if (triphase == false) {
        attributes = attributes.filter(a => a.triphase == false);
    }
    if (production == false) {
        attributes = attributes.filter(a => a.production == false);
    }
    return attributes.map(i => i.attribut).filter(a => a != null).filter(onlyUnique);
}


const indexEnergy = () => exposes.numeric('energy', ea.STATE).withUnit('Wh').withDescription('Index Base').withProperty('currentSummDelivered');

let exposesData = data.map(item => {
    if (item.unit != null && item.unit != '-' && item.unit != '') {
        return exposes.numeric(item.name, ea.STATE).withDescription(item.description).withProperty(item.attribut).withUnit(item.unit);
    } else {
        return exposes.text(item.name, ea.STATE).withDescription(item.description).withProperty(item.attribut);
    }

}).filter(e => e != null);


function parsingResponse(model, msg, entries) {
    let payload = {}
    for (let entry of entries) {
        if (msg.data[entry] != null) {
            payload[postfixWithEndpointName(entry, msg, model)] = precisionRound(msg.data[entry], 2)
        }
    }
    // console.log(payload);
    return payload;

}


const haMeterIdentificationFZ = {
    cluster: 'haMeterIdentification',
    type: ['attributeReport', 'readResponse', 'write'],
    convert: (model, msg, publish, options, meta) => {
        const standard = options && options.standard ? options.standard : false;
        const triphase = options && options.triphase ? options.triphase : false;
        const production = options && options.production ? options.production : false;
        return parsingResponse(model, msg, extractAttribute('haMeterIdentification', standard, triphase, production))
    }
};

const haElectricalMeasurementFZ = {
    cluster: 'haElectricalMeasurement',
    type: ['attributeReport', 'readResponse', 'write'],
    convert: (model, msg, publish, options, meta) => {
        const standard = options && options.standard ? options.standard : false;
        const triphase = options && options.triphase ? options.triphase : false;
        const production = options && options.production ? options.production : false;
        return parsingResponse(model, msg, extractAttribute('haElectricalMeasurement', standard, triphase, production))
    }
};

const seMeteringFZ = {
    cluster: 'seMetering',
    type: ['attributeReport', 'readResponse', 'write'],
    convert: (model, msg, publish, options, meta) => {
        const standard = options && options.standard ? options.standard : false;
        const triphase = options && options.triphase ? options.triphase : false;
        const production = options && options.production ? options.production : false;
        let payload = parsingResponse(model, msg, extractAttribute('seMetering', standard, triphase, production))
        //fusion HC/HP dans l'index commun
        if (payload['currentSummDelivered'] == 0 && (payload['currentTier1SummDelivered'] > 0 || payload['currentTier2SummDelivered'] > 0)) {
            payload['currentSummDelivered'] = payload['currentTier1SummDelivered'] + payload['currentTier2SummDelivered'];
        }
        return payload;
    }
};


async function poll(endpoint, standard, triphase, production) {
    // console.log('LINKY: read haMeterIdentification')
    let attrs = extractAttribute('haMeterIdentification', standard, triphase, production);
    let chunks = createChunk(attrs, 5);
    for (let chunk of chunks)
        await endpoint.read('haMeterIdentification', chunk);

    // console.log('LINKY: read haElectricalMeasurement')
    attrs = extractAttribute('haElectricalMeasurement', standard, triphase, production);
    chunks = createChunk(attrs, 5);
    for (let chunk of chunks)
        await endpoint.read('haElectricalMeasurement', chunk);

    // console.log('LINKY: read seMetering')
    attrs = extractAttribute('seMetering', standard, triphase, production)
    chunks = createChunk(attrs, 5);
    for (let chunk of chunks)
        await endpoint.read('seMetering', chunk);
}


const definition = {
    zigbeeModel: ['ZLinky_TIC'], // The model ID from: Device with modelID 'lumi.sens' is not supported.
    model: 'ZLinky_TIC', // Vendor model number, look on the device for a model number
    vendor: 'LiXee', // Vendor of the device (only used for documentation and startup logging)
    description: 'Lixee ZLinky', // Description of the device, copy from vendor site. (only used for documentation and startup logging)
    fromZigbee: [haMeterIdentificationFZ, seMeteringFZ, haElectricalMeasurementFZ],
    toZigbee: [],
    exposes: [indexEnergy(), ...exposesData],
    // The configure method below is needed to make the device reports on/off state changes
    // when the device is controlled manually through the button on it.
    configure: async (device, coordinatorEndpoint, logger) => {
        const SW1_ENDPOINT = 1;
        const endpoint = device.getEndpoint(SW1_ENDPOINT);

        endpoint.saveClusterAttributeKeyValue('haElectricalMeasurement', { acCurrentDivisor: 1, acCurrentMultiplier: 1 });
        endpoint.saveClusterAttributeKeyValue('seMetering', { divisor: 1, multiplier: 1 });
        //await poll(endpoint, false, false, false); // TODO recup les options ? 
        device.save();

    },
    options: [exposes.options.measurement_poll_interval(), option_standard(), option_triphase(), option_production()],
    onEvent: async (type, data, device, options) => {
        const endpoint = device.getEndpoint(1);
        if (type === 'stop') {
            clearInterval(globalStore.getValue(device, 'interval'));
            globalStore.clearValue(device, 'interval');
        } else if (!globalStore.hasValue(device, 'interval')) {
            const seconds = options && options.measurement_poll_interval ? options.measurement_poll_interval : 120;
            const standard = options && options.standard ? options.standard : false;
            const triphase = options && options.triphase ? options.triphase : false;
            const production = options && options.production ? options.production : false;
            await poll(endpoint, standard, triphase, production);
            const interval = setInterval(async () => {
                try {
                    await poll(endpoint, standard, triphase, production);
                } catch (error) {/* Do nothing*/ }
            }, seconds * 1000);
            globalStore.putValue(device, 'interval', interval);
        }
    },
    endpoint: (dev) => {
        return {
            'haElectricalMeasurement': 1, 'seMetering': 1, 'haMeterIdentification': 1
        }
    }
};

module.exports = definition;

[vk496]

Hi,

Thanks @djsyl !

I checked again and the problem was the version of the Z2M. When I updated to 1.22 all was working as expected (seems that edge versions don't update automatically, so switched to stable).

I suppose that in 1.21 the poll interval was completely ignored and used as real time.

[djsyl]

@vk496 vincent super si tout fonctionne bonne journée

[vk496]

Hello,

Even if my issue is solved, I still thinking that it would be better if the firmware report when something changes (and each N minutes, configurable) instead of poll from Z2M or any other Zigbee Controller. So I think is better to keep this issue opened for now

[fairecasoimeme]

this feature is already available for the reportables attributes. With zigbee, you can use the config report command to manage a report change with a value. But warning, all of this parameters can flood the zigbee network and provoque a power issue