Errorcontrol: Send data only when certain conditions are met?

[tophee]

I've been using your collect script for a number of months now to track the temperatures in a ventilation system and it work quite well but there is one problem that I've not been able to solve so far. And that is that for whatever reason, the script is sometimes receiving faulty data from the arduino (and by faulty data I don't just mean the values wrong field names).

Based on what I can tell from the log, what seems to happen is that for some reason one line of data is not completely transmitted (or received) so that the subsequent transmission is appended to the incomplete one. The subsequent transmission is also truncated but at the beginning, and I have no idea what is causing this (I'll put an example at the end of this post to make it easier to understand, but I don't think we need to get into the details). Could it have something to do with the scripts read timeout? Mine is set to 80 and the Arduino is transmitting two lines every 60 seconds (one right after the other)...

Anyway, I'm willing to accept that transmission sometimes fails and that I will lose those data points, so I just want those to be thrown away and not sent to Influxdb at all. How can I get the parser in the script to only send data to influx under certain conditions?

My idea would be that I include a code at the beginning of the line as well as at the end and the script should only send that line to influxdb if those codes match. If not, just discard the received data and wait for the next line. But I'm not sure whether and how this could be implemented...

---

Here is an example from the logs where I have highlighted the faulty field name (CR0entstate) and which the script "correctly" transmits to influxdb:

DEBUG:root:Received line 'FTX,status=testing elapsed=60001i,t0=-10.20,t1=14.30,t2=19.40,t3=-2.55,rh0=100.00,ah0=2.3,rh1=70.20,ah1=8.6,rh2=54.60,ah2=9.1,rh3=93.00,ah3=3.8,dew0=-10.2,dew1=8.9,dew2=10.0,dew3=-3.5,h-eff=92.8,t-eff=82.8,hum-gain=-7.88,CR0entstate=1,brightness=28i,desiredventstate=1,error0=0i,error1=0i,error2=0i,error3=0i,error4=0i,error5=0i,lederror=0i,timerup=2981332170i,timerdown=0i,timerhigh=0i,old_t_vals0=0i,old_t_vals1=0i,old_t_vals2=0i,old_t_vals3=0i,old_h_vals0=0i,old_h_vals1=0i,old_h_vals2=0i,old_h_vals3=0i,sensorerrors0=772i,sensorerrors1=1316i,sensorerrors2=1236i,sensorerrors3=1348i,h-index=5,end=2\r\n' DEBUG:root:Sending lines: ['FTX,status=testing elapsed=60001i,t0=-10.20,t1=14.30,t2=19.40,t3=-2.55,rh0=100.00,ah0=2.3,rh1=70.20,ah1=8.6,rh2=54.60,ah2=9.1,rh3=93.00,ah3=3.8,dew0=-10.2,dew1=8.9,dew2=10.0,dew3=-3.5,h-eff=92.8,t-eff=82.8,hum-gain=-7.88,CR0entstate=1,brightness=28i,desiredventstate=1,error0=0i,error1=0i,error2=0i,error3=0i,error4=0i,error5=0i,lederror=0i,timerup=2981332170i,timerdown=0i,timerhigh=0i,old_t_vals0=0i,old_t_vals1=0i,old_t_vals2=0i,old_t_vals3=0i,old_h_vals0=0i,old_h_vals1=0i,old_h_vals2=0i,old_h_vals3=0i,sensorerrors0=772i,sensorerrors1=1316i,sensorerrors2=1236i,sensorerrors3=1348i,h-index=5,end=2 1613287276238050816']

Up until CR0, the data is part of the first line of data. What comes after that seems to be the end of the second lone of data. Here is an example of what a correct transmission of both lines would look like:

DEBUG:root:Received line 'FTX,status=testing elapsed=60014i,t0=5.90,t1=18.60,t2=21.10,t3=9.90,rh0=100.00,ah0=7.2,rh1=54.70,ah1=8.7,rh2=49.50,ah2=9.1,rh3=91.00,ah3=8.5,dew0=5.9,dew1=9.3,dew2=10.1,dew3=8.5,h-eff=78.2,t-eff=83.6,hum-gain=-6.74,CR0-1=-1.00,CR0-2=-0.20,CR2-1=0.00,CR2-2=-0.00,end=1\r\n' DEBUG:root:Sending lines: ['FTX,status=testing elapsed=60014i,t0=5.90,t1=18.60,t2=21.10,t3=9.90,rh0=100.00,ah0=7.2,rh1=54.70,ah1=8.7,rh2=49.50,ah2=9.1,rh3=91.00,ah3=8.5,dew0=5.9,dew1=9.3,dew2=10.1,dew3=8.5,h-eff=78.2,t-eff=83.6,hum-gain=-6.74,CR0-1=-1.00,CR0-2=-0.20,CR2-1=0.00,CR2-2=-0.00,end=1 1617568674850982912'] DEBUG:root:Received line 'FTX_log ventstate=1,ledventstate=1,brightness=26i,desiredventstate=1,error0=0i,error1=0i,error2=0i,error3=0i,error4=0i,error5=0i,lederror=0i,timerup=1911202681i,timerdown=0i,timerhigh=0i,old_t_vals0=0i,old_t_vals1=0i,old_t_vals2=0i,old_t_vals3=0i,old_h_vals0=0i,old_h_vals1=0i,old_h_vals2=0i,old_h_vals3=0i,sensorerrors0=840i,sensorerrors1=780i,sensorerrors2=828i,sensorerrors3=896i,h-index=0,end=2\r\n' DEBUG:root:Sending lines: ['FTX_log ventstate=1,ledventstate=1,brightness=26i,desiredventstate=1,error0=0i,error1=0i,error2=0i,error3=0i,error4=0i,error5=0i,lederror=0i,timerup=1911202681i,timerdown=0i,timerhigh=0i,old_t_vals0=0i,old_t_vals1=0i,old_t_vals2=0i,old_t_vals3=0i,old_h_vals0=0i,old_h_vals1=0i,old_h_vals2=0i,old_h_vals3=0i,sensorerrors0=840i,sensorerrors1=780i,sensorerrors2=828i,sensorerrors3=896i,h-index=0,end=2 1617568675250469888']