[REQUEST] Water flow metering reading of IEC 62056-21 protocol smart water meters via photodiode/infrared optical eye reader header?

[Hedda]

Describe the problem you have/What new integration you would like

Like to get water flow metering from smart water meters with IR eye via hardware similar to Home Assistant Glow or SlimmeLezer.

Optical photodiode/infrared read and write header hardware for IEC 62056-21 protocol (formerly IEC 61107 / IEC 1107) like these:

https://www.mysensors.org/build/ir

http://jheyman.github.io/blog/pages/WirelessWaterMeter/

A hackerspace group in Denmark is selling these IR/photodiode optical reading heads for about $14 US (Google translate as test written in Danish) and they wrote a note that using magnets is required as the IR port transmitter / reader eye does not activate unless it sense a magnet:

https://wiki.hal9k.dk/projects/kamstrup

https://github.com/Hal9k-dk/kamstrup

I believe that a such IR reader adapter could be used with newer Kamstrup water meters with IR optical heads common in Europe.

Specifically, I myself live in Sweden where I believe that the Kamstrup MULTICA 21 / Kamstrup flowIQ 210x ultrasonic water meter is the most common model installed by the municipality's utility companies (e.i. local government agencies) in normal private residential houses here in Scandinavia (Sweden, Norway, Denmark, Finland, and Island) to track water usage for their service fees.

https://www.kamstrup.com/en-en/water-solutions/smart-water-meters

https://www.kamstrup.com/en-en/water-solutions/smart-water-meters/multical-21

They currently also have Kamstrup flowIQ 2200 "advanced model with acoustic leak detection":

and Kamstrup flowIQ 3100 "Commercial and industrial" model

https://www.kamstrup.com/en-en/water-solutions/smart-water-meters/flowiq-2200

https://www.kamstrup.com/en-en/water-solutions/smart-water-meters/flowiq-3100

Please describe your use case for this integration and alternatives you've tried:

I believe many new water meters offer such two-way bidirectional infrared port (IR receiver and sender header) for communication?

https://www.mysensors.org/build/ir

http://jheyman.github.io/blog/pages/WirelessWaterMeter/

Kamstrup seems to only provide some technical documentation about all their products here but nothing about their protocols:

https://products.kamstrup.com/index.php

Kamstrup apparently calls their DLMS implementation for Kamstrup Meter Protocol (KMP) it has been reverse-engineered here:

https://github.com/bsdphk/PyDLMS

https://github.com/RuntimeError123/hass-mc66c

https://github.com/KenanV/KamstrupMultical66

https://github.com/Hal9k-dk/kamstrup/tree/master/Software%20eksempler/kamstrup_multical402

https://github.com/Hal9k-dk/kamstrup/tree/master/Software%20eksempler/kamstrup_powermeter

https://github.com/bsdphk/PyKamstrup

https://frack.nl/wiki/Stadsverwarming

https://forum.mysensors.org/topic/3525/district-heating-city-heating-stadsverwarming-mysensor-ir-sender-receiver/

https://www.domoticz.com/forum/viewtopic.php?t=11333

As I understand most of those optical probes should work with optical infrared waves be compliant with the international standard IEC 62056-21 (formerly IEC 61107 / IEC 1107) or ANSI C12.18 communications protocols for energy metering for reading utility meters?

https://github.com/pwitab/iec62056-21

https://en.wikipedia.org/wiki/IEC_62056

https://en.wikipedia.org/wiki/Smart_meter#Protocols

Kamstrup official (and expensive) optical read-out head tools are called 6699-099 (USB) and 699-102 (9F D-sub plug serial)

https://www.termonet.rs/pdf/Kamstrup/Optical%20Read-out%20(IR)%20Head%20-%20Data%20Sheet%20-%20English.pdf

Additional context

FYI, most of the older and newer Kamstrup electricity meters look to feature a similar or same type of bi-directional optical header, so the hardware could probably be repurposed for other meters as well

PS: Off-topic but FYI; some if not all of these models of Kamstrup water meters can also be read remotely via Sigfox network RF (Radio Frequency) but that communication traffic is most of the time encrypted, at least if the water meter itself is owned and operated by a utility company or a service provider.

https://www.kamstrup.com/en-en/water-solutions/water-meter-reading

https://www.kamstrup.com/en-en/water-solutions/water-meter-reading/usb-meter-reader

If the case is that the RF radio communication is not encrypted then there look to be projects that can work with that:

https://github.com/adams-okode/kamstrup-integration

https://github.com/weetmuts/wmbusmeters

https://github.com/tobiasrask/wmbus-client

[Hedda]

This might be related the the request for SML (Smart Message Language) protocol for reading smartmeter?

https://github.com/esphome/feature-requests/issues/1041

There is a lot collected about workings of SML (a.k.a. "D0 interface") smartmeters with IR two-way here:

https://github.com/volkszaehler/libsml

https://www.msxfaq.de/sonst/bastelbude/smartmeter_d0_sml.htm

[ajvdw]

Have a look at my solution for reading pulse from a sensus620 watermeter. With minor adjustments you can use the same hardware solution....

[Hedda]

@ajvdw These mentioned Kamstrup MULTICA 21 / Kamstrup flowIQ meters don't generate a pulse that represents a metering value.

IR port on these Kamstrup MULTICA 21 / Kamstrup flowIQ communicate via a bi-directional protocol over that IR port (serial-based?).

You can not compare these to for example Kamstrup electricity meters (like their OMNIPOWER meters) which have both a pulse diod where the pulses can just be counted as well as the more complex IR port for their bi-directional protocol similar to MULTICA/flowIQ:

See example this project about building IR sender and receiver hardware to both read and write https://wiki.hal9k.dk/projects/kamstrup

[ajvdw]

It's not about the pulse. The hardware that I have designed can easily be adjusted to communicate via IR. It already contains an IR sender and receiver. You only have to connect the IR sender to a datapin in order to send data to the Kamstrup. Receiving is already possible. Then you can use softserial to implement the protocol.

Op ma 25 okt. 2021 om 15:19 schreef Hedda @.***>:

@ajvdw https://github.com/ajvdw These mentioned Kamstrup MULTICA 21 / Kamstrup flowIQ meters don't generate a pulse that represents a metering value.

IR port on these Kamstrup MULTICA 21 / Kamstrup flowIQ communicate via a bi-directional protocol over that IR port (serial-based?).

You can not compare these to for example Kamstrup electricity meters (like their OMNIPOWER meters) which have both a pulse diod where the pulses can just be counted as well as the more complex IR port for their bi-directional protocol similar to MULTICA/flowIQ:

[image: image] https://user-images.githubusercontent.com/6320001/138702295-e57ef8a1-d42b-4794-93eb-f4719de0433d.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/esphome/feature-requests/issues/1402#issuecomment-950921427, or unsubscribe https://github.com/notifications/unsubscribe-auth/AET5LMIIVOXSZFQDOE2XXWLUIVKMLANCNFSM5DULRVRA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Met vriendelijke groeten,

Arie Johan van de Werken

[coltju]

Check out https://github.com/golles/Home-Assistant-Sensor-MC66C

Im using this for my block heating sensor, which uses the same tech.

[evlo]

Hi, i'm looking at projects regarding this, either esphome (preferably)

OP did nice summary of the things I also found.

What i understand that IEC 62056-21 is quite widely used in this environment, and sometimes called "S0" (probably something from DIN). I even seems basically UART RS485, just trough optical IR diode(s). BTW I remember ages ago IR was used by laptops and cellphones to transfer data.

I think not a lot people know about this and all the projects I found so far about this are quite old/cumbersome or me or at least quite hard to follow.

So I was wondering if i understand it correctly that all need is this https://www.aliexpress.com/item/33005112599.html - connect it using "dupont" cables to nodemcu32 ana i should be able to read the data if speed set to 9600 baud or slower?

https://onemeter.com/cs/specifications/ found this, not esp, but maybe they just convert ir uart to bt uart? https://www.fiedler.company/sites/default/files/dokumenty/cenik_2022_v104b.pdf here they sell 3d printed assembled optical diodes with correct IEC 62056-21 4.3.2 magnetic ring etc. https://www.fiedler.company/en/products/smart-metering-remote-meter-readings/sensors-and-transducers-meters/elm1-converter-electric (suspiciously simillar to https://wiki.hal9k.dk/projects/kamstrup solution ...)

EDIT: found this https://www.aliexpress.com/item/1005003509520122.html but it is USB

this one is RS232 https://www.aliexpress.com/item/1005003328350869.html

[evlo]

I think the main issue is sending /?!\r\n / HEX: 2F3F210D0A to poll the meter, otherwise maybe the default SML component might work.

[evlo]

Most likely what my meter spits out is not actually SML, but called IEC62056-21 or sometimes d0 - i think it goes like d0 is the port, IEC62056-21 is the definition of the IR communication, SML is the "extended" protocol and OBIS codes are what each code mean, so even though meter uses IEC62056-21, d0 and OBIS it is most likely not SML, but just IEC62056-21 compliant (which contains basic protocol, but not exactly SML - sml seems to me that it might be IEC62056-21 for multiple meters on the same bus)

This is used by more meters then just water, very similar to SML, but i think separate component is need for easiest implementation. What I wrote in my previous post about using SML component is not true

example:

[jakobschou]

Hopefully somebody could figure out to get some cheap optical probe like this to work: https://a.aliexpress.com/_mK4qGxM so we dont need to solder something ourselves.

It seems that it is possible to do with other Kamstrup meters: https://github.com/golles/ha-kamstrup_403 (Somebody is using https://a.aliexpress.com/_mPY7OYe and getting it to work, though there needs to be attached magnet aswell https://a.aliexpress.com/_mOYY7d4 to get the IR sensors working on the Kamstrup meter)

Though this setup is using pykamstrup (https://github.com/golles/ha-kamstrup_403/tree/main/custom_components/kamstrup_403) which I dont know if it is the right/same protocol for water meters with IEC62056-21, but maybe it could be changed to pyDLMS (https://github.com/bsdphk/PyDLMS). Here is also some code for the old IEC1107 protocol which maybe could be useful, since the original Kamstrup Optical read-out head seems to be only compliant with this protocol...

Hopefully somebody could fork kamstrup_403 and make it work with Kamstrup Multical 21/FlowIQ 2200 - I just dont have the knowledge to do so.

[evlo]

maybe take a look at

• https://github.com/evlo/esphome_ZPA-ZE312
• https://github.com/jplitza/esphome_components

using pretty much same thing

[cfeenstra1024]

If this meter uses the 'Kamstrup Meter Protocol' (KMP) we might be able to use the Kamstrup Multical40x component for ESPHome I've created recently.

If you have the hardware to connect an ESP to the optical interface, I can create a version for ESPHome that can be used to request available parameters. You can then test with your meter. If that works, I can extend the ESPHome component functionality.

I don't have this meter myself, but I do have the Kamstrup Multical 403 (district heating meter) which works well with the created component in ESPHome. I noticed that both meters mention a 'METERTOOL HCM' that can be used with the optical interface in their data sheets. This makes me think they might use the same protocol.

Some info about the component:

• ESPHome component for Kamstrup multical 40x
• Electronics and housing I've created for use with the Multical 40x
• Documentation

[onkelbeh]

I just printed an adapter to hold an IR adapter on the meter, I am going to give it a try soon.

[glance-]

I can report that I've had success talking to a flow IQ 2200 using https://github.com/ErikErnst/kamstrup and 1200 baud.

I haven't found which variables the water meter uses yet, but I have read the time / date and serial number from the meter using a ir eye.

[jakobschou]

I can report that I've had success talking to a flow IQ 2200 using https://github.com/ErikErnst/kamstrup and 1200 baud.

I haven't found which variables the water meter uses yet, but I have read the time / date and serial number from the meter using a ir eye.

What IR interface did you use?

[glance-]

Some random one from flee-bay a couple of years back. The only issue I've found is that the physical form factor of flow IQ 2200 isn't the "standard" one so i had to do some tail then error and when it finally worked apply some tape to keep the ir eye on place. "Normal" devices have a embedded metal plate and some registring features to keep the ir eye in the right place.

[copystring]

You can send 803f100100444dc00d which the meter responds for example with 403F1000442804430000215DA390 In my case the meter has a volume of 8.541m³ which is 215D. To get this number, take 0000215D and convert to decimal (8541). Divide it by 1000, and you get the correct volume of 8.541

[copystring]

Sending 803f100100444dc00d can be split like this: | start byte | destination address | CID | requested var | CRC | end byte | | 80 | 3f10 | 01 | 0044 | 4dc0 | 0d |

on the receiving side: | start byte | destination address | requested var | don't know what this is | data | CRC | end byte | | 40 | 3F10 | 0044 | 280443 | 0000215D | A390 | 0d |

where 0044 stands for the current meter value

[aquaticus]

Hi All,

I created a component for IEC62056-21 meters. It can exchange data with utility meters. Mostly for electricity but also water, thermal and other meters.

It supports both bidirectional and unidirectional communication.

• bidirectional - you need optical interface with IR diode and phototransistor that can send and receive data. This is for modes A,B, and C.
• unidirectional - only IR phototransistor is required. This is for mode D - meter periodically sends data to the interface.

The component does not support binary encoded data (HDLC)..

Hardware required: Optical interface connected to serial port.

IEC 62056-21 term is used for multiple protocols (with the same hardware layer but different data encoding). The component supports meters that provides ASCII encoded data, something like that:

1-0:15.8.1(00000009999.567*kWh)
1-0:15.8.2(00000000000.000*kWh)
1-0:15.8.3(00000000000.000*kWh)
1-0:15.8.4(00000000000.000*kWh)

Detailed description of the component: https://aquaticus.info/iec62056.html

PR to ESPHome: https://github.com/esphome/esphome/pull/4388

[copystring]

Hi @aquaticus,

does this support only obis codes?

[aquaticus]

@copystring Not sure I understand the question. The component can only interpret information sent as OBIS codes.

[copystring]

Correct me if I'm wrong, but the way I see it, the Multical 21 does not use OBIS codes.

[aquaticus]

Just realized Multical 21 probably does not use IEC 62056-21 protocol. I was a bit mislead by the log in one of the comments. In that case the component cannot be used with Multical. And yeah, it seems Multical uses different data model than OBIS.

[glance-]

where 0044 stands for the current meter value

I can attest for this. This works also by using readvar from https://github.com/ErikErnst/kamstrup and asking for value 68 (0x44).

280443 is decoded as 0x28 means m^3, 04 means 4 byte value and 43 means how to convert the value, aka 0x40 it's a negative exponent, and 0x03 it should be ^-3 , all according to above referenced code.

[glance-]

Btw, 74 (0x4A) gives you l/h.

[jakobschou]

@glance- So to understand - have you figured out how to connect to the meter with the IR-to-USB, and get readings from it, that can be pushed to etc. Home Assistant?

[jakobschou]

I have found this fork that should work with the ir-to-usb with HA:

https://github.com/kristianschneider/ha-multical_21

[mroek]

@glance- I am very interested in your findings, as I also have a flowIQ 2200 in my house (installed by the municipality). The meter sends data using Wireless M-bus, but unfortulately it is encrypted, and the municipality does not want to provide me with the encryption key.

My question to you is whether you know if the data obtained through the optical eye is also going to be encrypted in my case. Do you know whether your meter also sends encrypted data wirelessly? If so, it probably means that the optical data is not encrypted.

Second question is if you have verified that you can poll the meter whenever you want, or if it imposes some limit on the optical interface to preserve battery life?

Third question is if you know whether a magnet is necessary to activate the optical interface on the meter or not. There is no metal plate for the magnet to attach to, so the official Kamstrup optical eye needs a plastic bracket that attaches to the meter to hold the optical eye in place. It could still need a magnet to activate the optical interface, though.

I guess you just used a Linux-machine to run the software you linked to (https://github.com/ErikErnst/kamstrup), but I suppose it should be possible to adapt the necessary parts of the software to run on an ESP to provide a wireless reader of sorts.

[mroek]

I have a flowIQ2200 (like @glance- ), but there is absolutely no way I can get it to talk to me. I've just DIYed a reading head (3D-printed multiple brackets for the IR leds), and I have added magnets in all kinds of orientations, but the optical interface is just completely dead.

The IR-leds I used was just something I had lying around, and I don't know their peak wavelength, but I doubt that the problem is there. If I point my IR-leds at each other, I will receive a correct echo in a serial terminal, so the circuit (even if just cobbled together on a breadbord) seems fine.

Kamstrup sells a bracket for mounting their optical eye, and due to the design of the meter, the optical eye doesn't really touch the face of the meter, instead it rests against the outer rim. I assume there are light guides in the braket to ensure a good optical connection to the meter. What this also means, is that the magnet sensor in the meter must be located inside the outer rim, and I am fairly certain this is correct. Reason is I found this text in a Kamstrup document:

It is possible to toggle through all views by means of the Kamstrup meter reader head, part no. 6699- 099, placed on the optical eye at the meter front. For more details on the meter reader head/optical eye, please see chapter 11 Tools and programs. The magnet needs to be removed, and applied again on the optical eye, to switch display view.

I found a position for the magnet where I could get the meter to change the views, so it seems natural to think that this is also the position used to turn on the optical port. However, still nothing. Very frustrating.

There is also a statement that the polarity of the magnet is important, and I did notice that the above mentioned change of display views only works with the magnet oriented with the south pole towards the meter. This is opposite to what the 62056-21 standard says (for the reading head):

Magnetization: axial, north pole directed towards the tariff device.

It seems unlikely that my meter has a permanently disabled optical port, I can't find any order codes for such an option. And just to make it clear, I am sending KMP telegrams, like mentioned by @copystring further up in this discussion.

Edit: Another curious thing that I just discovered is that if I just start a loop of readings without any magnet attached, and then place the magnet on the spot where I can get it to change the display view, the displacy just briefly flickers to another view and then returns to normal. This is a clear indication that it does register something at the IR port, because if I don't send anything, the display views can be cycled as described in the tech doc.

[mroek]

Ok, so I finally have success! Turns out that there was indeed a problem with my receiver circuit. It wasn't sensitive enough to trigger on the IR signal from the meter.

I came up with the idea to have only my TX LED connected and sending repeated requests, and then I pointed my phone camera to the TX led of the meter. It showed something was sent, which of course prompted me to increase the sensitivity of the receiver circuit.

After that, I did get sensible data from the meter. Would be nice to have an ID for some more registers, because there are plenty of them. There's a long list in that Kamstrup document I found, but it just names them, there's no telling the address/ID.

[mroek]

I left my reader polling at 30 seconds interval, and at some point (maybe a couple of hours) the meter stopped responding. :-(

Not sure if there is some internal counter/timer to limit readouts over the optical interface to preserve battery, but it is still bad news, as it means it is not a viable method to monitor the water consumption over time. Unless of course there is an acceptable polling rate that the meter will tolerate. I also noticed that it would shut down the optical interface if it wasn't polled, even if the magnet was left in place.

Will be interesting to see if it starts responding again tomorrow, but it would be really bad design if it doesn't.