rtl_433
rtl_433 copied to clipboard
merbanan
Lux and UV incorrect for Cotech 36-7959
UV and light_lux were added for Cotech 36-7959 in commit 7f14007 , but light_lux is not decoded correctly. Maximum decoded light_lux value in decoded samples is always 69627 and I'm getting the same value although the console itself properly displays higher values and the value should go up to 200.000.
It was added from #1575 with 67ee1d6 and is 16 (or 17?) bit, noted to reach 200.000? https://github.com/merbanan/rtl_433/blob/master/src/devices/cotech_36_7959.c#L42 This looks strange
int flags = (b[7] & 0xf0) >> 4; // [56:4]
int light_lux = (b[10] << 8) + b[11] + ((flags & 0x08) << 9); // [80:16]
So top bit of b[7] goes to bit 12? I guess it should be 16, i.e.
int light_lux = (b[10] << 8) | b[11] | ((b[7] & 0x80) << 9); // [56:1][80:16]
Can you modify your source code, recompile and test?
@andrewpile can you double-check that line?
Recompiled, will test and report back tomorrow when the sun comes up to see how it behaves. I've tried some test samples, but they don't look promising. While one of my samples does decode over 69627 (77661 to be exact), all my other samples had lower values and all the samples in rtl_433_tests now decode as 131067 instead of 69627, which is suspicious.
Additionally, it seems that UV index is scaled by 10 so it might be a good idea to map it as a float, but I'll test this out as well.
It's been a long time since I looked at this, but the origins of this code were always a little suspicious and undocumented, so I suspect it needs more research.
As expected, lux values are now capped at 131069, all those below are properly decoded.
time : 2022-05-18 12:41:48
model : Cotech-367959 ID : 28
Battery : 1 Temperature: 66.6 F Humidity : 45 % Rain : 9.3 mm Wind direction: 85 Wind : 2.0 m/s Gust : 3.0 m/s Light Intensity: 78155 lux
UV Index : 4.0 Integrity : CRC
time : 2022-05-18 12:42:52
model : Cotech-367959 ID : 28
Battery : 1 Temperature: 67.1 F Humidity : 45 % Rain : 9.3 mm Wind direction: 82 Wind : 1.8 m/s Gust : 2.3 m/s Light Intensity: 131069 lux
UV Index : 6.9 Integrity : CRC
The lux value is not a 16-bit integer. We do have a few spare bits of unknown purpose (block J), could they belong here? My knowledge of C and bitwise operation is limited, how would I go about testing this?
Record some samples and their corresponding reading from the official receiver. Then we can look for what bits belong to what value.
Even better add decoder_log_bitrow(decoder, 0, __func__, b, sizeof(b) * 8, "bitrow"); at https://github.com/merbanan/rtl_433/blob/master/src/devices/cotech_36_7959.c#L85 (the line before // Extract data from buffer), recompile and run.
Note the decoded rows you get and the light values you expect.
The console displays higher values as klux, the following samples should be either 164.4 klux or 164.9 klux (although I'm not absolutely certain about the last one).
cotech_36_7959_decode: bitrow: {112} c1 c0 0c 11 5a 00 5d 84 37 2c ff fd 45 b2
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
time : @0.324652s
model : Cotech-367959 ID : 28
Battery : 1 Temperature: 67.9 F Humidity : 44 % Rain : 9.3 mm Wind direction: 90 Wind : 1.2 m/s Gust : 1.7 m/s Light Intensity: 131069 lux
UV Index : 6.9 Integrity : CRC
cotech_36_7959_decode: bitrow: {112} c1 c0 04 07 20 00 5d 84 3a 2d ff fd 45 e6
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
time : @0.193760s
model : Cotech-367959 ID : 28
Battery : 1 Temperature: 68.2 F Humidity : 45 % Rain : 9.3 mm Wind direction: 32 Wind : 0.4 m/s Gust : 0.7 m/s Light Intensity: 131069 lux
UV Index : 6.9 Integrity : CRC
cotech_36_7959_decode: bitrow: {112} c1 c4 08 0a 37 00 5d 84 3d 2b ff fd 44 c7
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
time : @0.325144s
model : Cotech-367959 ID : 28
Battery : 1 Temperature: 68.5 F Humidity : 43 % Rain : 9.3 mm Wind direction: 311 Wind : 0.8 m/s Gust : 1.0 m/s Light Intensity: 131069 lux
UV Index : 6.8 Integrity : CRC
Before I forget, the UV value is indeed scaled by 10, ranges from 0 to 150, so it should be formatted as a float to avoid confusion (to prevent UV 0.9 being interpreted as UV 9). The original display console rounds this value up and only displays 0-15.
The value of ff fd (b[10] b[11]) does look more like an error code than a reading.
Can you grab values over a wider range and paste lines this here?
- {112} c1 c0 0c 11 5a 00 5d 84 37 2c ff fd 45 b2 : 164.4 klux or 164.9 klux
I'll see what I can do, might take a while because they have to be in a precise range. I have a bunch of eligible samples, but not their expected values.
Here's a few I managed to force with a flashlight, they're the best I can do today.
{112} c1 c0 0c 14 12 00 5d 84 47 1f ff fd 22 8b : probably 147.7 klux
{112} c1 c0 1c 21 3d 00 5d 84 48 1e ff fd 1f 3c : probably 146.2 klux
{112} c1 c0 16 21 48 00 5d 84 48 1e ff fd 1f 10 : possibly 146.2 klux
{112} c1 c0 0d 11 8b 00 5d 84 48 1d ff fd 1b 66 : 144.2 klux
{112} c1 c0 02 03 4a 00 5d 84 4a 1e ff fd 20 df : 146.7 klux
{112} c1 c0 00 03 69 00 5d 84 4c 1e ff fd 18 7d : 142.8 klux
{112} c1 c4 07 14 3d 00 5d 84 4d 1d ff fd 13 80 : 140.3 klux
{112} c1 c4 13 17 32 00 5d 84 4e 1d ff fd 12 9a : 139.8 klux
{112} c1 c4 0f 14 39 00 5d 84 4c 1d ff fd 1e 0e : 145.7 klux
{112} c1 c0 09 0d 04 00 5d 84 4d 1e ff fd 19 f0 : 143.3 klux
If you need values that properly decode across the entire range, I have plenty of that.
{112} c1 c0 16 1e 6e 00 5d 84 29 2c e8 ba 37 1b : 124812 lux
{112} c1 c0 1a 28 5c 00 5d 84 24 2c e8 4d 32 43 : 125005 lux
{112} c1 c0 1f 28 47 00 5d 84 22 2c fc ce 2f 63 : 130254 lux
{112} c1 c0 14 21 45 00 5d 84 16 3d 22 75 1a 0b : 74357 lux
{112} c1 c0 07 0d 2c 00 5d 84 13 3d 17 b5 14 60 : 71605 lux
{112} c1 c0 19 2b 44 00 5d 84 11 3d 01 3c 14 2f : 65852 lux
Oh, those all have ff fd in the bytes we expect the light value to be in.
Here is a BitBench with the codes. I don't see where the light value might be? It just seems to be "overflow".
If it's overflowing, there's a chance that the light value is broadcast in a different message?
Am I completely misreading this or is there a different package before the main one which we decode?
rtl_433 -X "n=Cotech-TEST,m=OOK_MC_ZEROBIT,s=488,l=0,r=988" -Y minsnr=20 -S all
time : 2022-05-18 20:33:48
model : Cotech-TEST count : 1 num_rows : 1 rows :
len : 127 data : 0029838000006600ba07e04600960080
codes : {127}0029838000006600ba07e04600960080
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
time : 2022-05-18 20:33:48
model : Cotech-TEST count : 1 num_rows : 1 rows :
len : 128 data : 0014c1c0000033005d03f023004b0040
codes : {128}0014c1c0000033005d03f023004b0040
cotech_36_7959_decode: bitrow: {112} c1 c0 00 00 33 00 5d 03 f0 23 00 4b 00 40
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
time : 2022-05-18 20:33:48
model : Cotech-367959 ID : 28
Battery : 1 Temperature: 60.8 F Humidity : 35 % Rain : 9.3 mm Wind direction: 51 Wind : 0.0 m/s Gust : 0.0 m/s Light Intensity: 75 lux
UV Index : 0.0 Integrity : CRC
cotech_36_7959_decode: bitrow: {112} c1 c0 00 00 33 00 5d 03 f0 23 00 4b 00 40
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
time : 2022-05-18 20:33:48
model : Cotech-367959 ID : 28
Battery : 1 Temperature: 60.8 F Humidity : 35 % Rain : 9.3 mm Wind direction: 51 Wind : 0.0 m/s Gust : 0.0 m/s Light Intensity: 75 lux
UV Index : 0.0 Integrity : CRC
*** Saving signal to file g001_433.92M_250k.cu8 (99015 samples, 262144 bytes)
Yes, in the spectrogram (https://triq.org/pdv/) and looking at the data it very much looks like a proper transmission. And it's already a valid Manchester decode. Can you capture more of those?
Oh and the original author of the decoder knew that too ;) https://github.com/merbanan/rtl_433/blob/master/src/devices/cotech_36_7959.c#L57-L62
Yes, in the spectrogram (https://triq.org/pdv/) and looking at the data it very much looks like a proper transmission. And it's already a valid Manchester decode. Can you capture more of those?
Not a problem capturing them, I have them from my tests above (when light_lux goes into overflow). I have a sneaking suspicion that this package might also contain the watts per square metre because the display console can display both. I initially thought that it just approximated the w/m2 value from the illumination using a fixed conversion factor, but the conversion factor varied between readings.
Here's that batch, in transmission order, along with the expected values of klux.
{127}002983801a231600bb08903bfffa36cc
{112} c1 c0 0d 11 8b 00 5d 84 48 1d ff fd 1b 66 : 144.2 klux
{127}0029838004069400bb08943dfffa41be
{112} c1 c0 02 03 4a 00 5d 84 4a 1e ff fd 20 df : 146.7 klux
{127}002983800006d200bb08983dfffa30fa
{112} c1 c0 00 03 69 00 5d 84 4c 1e ff fd 18 7d : 142.8 klux
{127}00298388262e6400bb089c3bfffa2534
{112} c1 c4 13 17 32 00 5d 84 4e 1d ff fd 12 9a : 139.8 klux
{127}002983881e287200bb08983bfffa3c1c
{112} c1 c4 0f 14 39 00 5d 84 4c 1d ff fd 1e 0e : 145.7 klux
{127}00298380121a0800bb089a3dfffa33e0
{112} c1 c0 09 0d 04 00 5d 84 4d 1e ff fd 19 f0 : 143.3 klux
EDIT: corrected second value.
Sorry, false alarm. Those are identical packets, just shifted one bit. I should have checked that first.
So no second transmission. Only the ff fd you see on overflow.
Okay, how about this? On every other (unrepeated) package, I get the 25-28 extra rows:
Detected OOK package 2022-05-19 12:04:13
cotech_36_7959_decode: bitrow: {112} c1 c0 07 0a 5a 00 5d 84 51 1d ff fd 2c d7
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
time : 2022-05-19 12:04:13
model : Cotech-367959 ID : 28
Battery : 1 Temperature: 70.5 F Humidity : 29 % Rain : 9.3 mm Wind direction: 90 Wind : 0.7 m/s Gust : 1.0 m/s Light Intensity: 131069 lux
UV Index : 4.4 Integrity : CRC
Analyzing pulses...
Total count: 102, width: 123.55 ms (30887 S)
Pulse width distribution:
[ 0] count: 77, width: 504 us [472;532] ( 126 S)
[ 1] count: 25, width: 992 us [984;1000] ( 248 S)
Gap width distribution:
[ 0] count: 76, width: 472 us [448;496] ( 118 S)
[ 1] count: 25, width: 956 us [948;964] ( 239 S)
Pulse period distribution:
[ 0] count: 65, width: 976 us [964;992] ( 244 S)
[ 1] count: 22, width: 1464 us [1444;1488] ( 366 S)
[ 2] count: 14, width: 1948 us [1944;1956] ( 487 S)
Pulse timing distribution:
[ 0] count: 153, width: 488 us [448;532] ( 122 S)
[ 1] count: 50, width: 972 us [948;1000] ( 243 S)
[ 2] count: 1, width: 10004 us [10004;10004] (2501 S)
Level estimates [high, low]: 15956, 671
RSSI: -0.1 dB SNR: 13.8 dB Noise: -13.9 dB
Frequency offsets [F1, F2]: -1676, 0 (-6.4 kHz, +0.0 kHz)
Guessing modulation: Manchester coding
view at https://triq.org/pdv/#AAB10301E803CC271480808080808080808191918090818080808090808180808080808080808080908081808080919180919081918080808080808080809190808190818080809180809191808091808090808190808080808080808080808080808191809190818090819190808255
Attempting demodulation... short_width: 504, long_width: 0, reset_limit: 968, sync_width: 0
Use a flex decoder with -X 'n=name,m=OOK_MC_ZEROBIT,s=504,l=0,r=968'
pulse_slicer_manchester_zerobit(): Analyzer Device
bitbuffer:: Number of rows: 1
[00] {127} 00 29 83 80 0e 14 b4 00 bb 08 a2 3b ff fa 59 ae
*** signal_start = 6390574, signal_end = 6441455, signal_len = 50881, pulses_found = 102
Iteration 1. t: 178 min: 117 (77) max: 239 (25) delta 65
Iteration 2. t: 178 min: 117 (77) max: 239 (25) delta 0
Pulse coding: Short pulse length 117 - Long pulse length 239
Short distance: 122, long distance: 130, packet distance: 248
p_limit: 178
bitbuffer:: Number of rows: 26
[00] { 9} 00 00 : 00000000 0
[01] { 1} 80 : 1
[02] { 1} 80 : 1
[03] { 3} 40 : 010
[04] { 7} 08 : 0000100
[05] {13} 00 20 : 00000000 00100
[06] { 4} 10 : 0001
[07] { 1} 80 : 1
[08] { 2} 40 : 01
[09] { 2} 80 : 10
[10] { 1} 80 : 1
[11] {10} 00 40 : 00000000 01
[12] { 3} 80 : 100
[13] { 2} 80 : 10
[14] { 4} 10 : 0001
[15] { 3} 20 : 001
[16] { 1} 80 : 1
[17] { 3} 20 : 001
[18] { 5} 20 : 00100
[19] {15} 80 00 : 10000000 0000000
[20] { 1} 80 : 1
[21] { 2} 40 : 01
[22] { 2} 80 : 10
[23] { 3} 40 : 010
[24] { 1} 80 : 1
[25] { 3} 80 : 100
*** Saving signal to file g013_433.92M_250k.cu8 (66156 samples, 262144 bytes)
Ignore, it's just a wrong guess by the analyzer. This is getting really weird. The data has to be somewhere in the package because it gets regularly updated every 15 seconds along with all other data.
klux is derived from the UV value. I played around with it a bit and found that the formula (131000 + (UV * 491) ) / 1000 gets the same klux values as reported (after rounding). The constants could be further tuned or confirmed with values closer to the point of lux overflow.
It's certainly good to extract what the official receiver sees. This rant is slightly off (illuminance vs irradiance, rather than illuminance vs UV index), but might be useful in understanding that converting UV to lux is an unsound hack, even if equipment manufacturers are willing to do it.
https://github.com/weewx/weewx/wiki/Watts-and-lux
Interesting! Yeah, it’s likely a hack, but personally knowing nothing about the properties of the sensors I’d want more data before deciding how bad of a hack it is. It would be interesting to see how a camera lens UV filter in front of the sensors affected the readings.
I'll test this out, I should have an UV filter somewhere.
It's certainly good to extract what the official receiver sees. This rant is slightly off (illuminance vs irradiance, rather than illuminance vs UV index), but might be useful in understanding that converting UV to lux is an unsound hack, even if equipment manufacturers are willing to do it.
As far as I can see, there's actually just the UV sensor in the window on the integrated sensor unit and, to be fair, the manufacturer does specify in the manual that the accuracy and resolution for UV index are ±1, while "sunlight" has ±15% accuracy and resolution.
Additionally, this would explain why I was unable to establish a fixed conversion factor between w/m2 and lux. Rounding errors.
In any case, what would be our best course of action in regards to RTL_433? Should it just report the (likely derived) lux value until it overflows and stop reporting it while it's capped, with a note somewhere in the documentation that it's derived and how to do so, thus letting the end-recipient of the data (weewx, for example) deal with approximations upwards from 131 klux? I don't think we should ignore this value altogether because it's bound to raise issues in the future, but we also can't let it report "capped" values. Is there an official stance for situations such as this?
Matching the official receiver is fine, unless it does really strange things.
But do I get it right that when the sensor maxes out it just makes up values together with the UV index ?
The receiver probably just rounds the received lux value (derived by the sensor unit from UV) to four digits until it maxes out, after that it does the calculations on its own. I assume my version of the receiver derives values for w/m2 and lm/ft2 directly from the UV index and doesn't bother with lux.