How to dismiss incomple received lines?

[tophee]

I have been using your script for a few years now and it did me great service. Thanks for creating it.

Recently I had to change my setup and I can't seem to get the script running continuously on the new machine. At first, it runs fine, but after a few minutes up to a few hours, it gets stuck with this error:

DEBUG:root:Received line b'FTX,status=testing elapsed=60017i,t0=17.90,t1=19.20,t2=22.90,t3=22.20,rh0=91.20,ah0=13.9,rh1=100.00,ah1=16.5,rh2=70.20,ah2=14.3,rh3=83.30,ah3=16.4,dew0=16.4,dew1=19.2,dew2=17.2,dew3=19.2,h-eff=601.1,t-eff=26.0,hum-gain=34.64,CR0-1=-0.20,CR0-2=-0.20,CR2-1=0.00,CR2-2=0.20,end=1\r\n'
DEBUG:root:Sending lines: [b'FTX,status=testing elapsed=60017i,t0=17.90,t1=19.20,t2=22.90,t3=22.20,rh0=91.20,ah0=13.9,rh1=100.00,ah1=16.5,rh2=70.20,ah2=14.3,rh3=83.30,ah3=16.4,dew0=16.4,dew1=19.2,dew2=17.2,dew3=19.2,h-eff=601.1,t-eff=26.0,hum-gain=34.64,CR0-1=-0.20,CR0-2=-0.20,CR2-1=0.00,CR2-2=0.20,end=1 1695087145207804928']
DEBUG:root:Received line b'FTX_log ventstate=1,ledventstate=1,buttonpressed=0i,ledventstatechanged=0i,brightness=25i,desiredventstate=1,error0=0i,error1=0i,error2=0i,error3=0i,error4=0i,error5=0i,lederror=0i,printerror=0i,timerup=20464546i,timerdown=0i,timerhigh=0i,old_t_vals0=0i,old_t_vals1=0i,old_t_vals2=0i,old_t_vals3=0i,old_h_vals0=0i,old_h_vals1=0i,old_h_vals2=0i,old_h_vals3=0i,senFTX,status=testing elapsed=60011i,t0=17.90,t1=19.20,t2=22.89,t3=22.20,rh0=91.20,ah0=13.9,rh1=100.00,ah1=16.5,rh2=70.20,ah2=14.3,rh3=83.30,ah3=16.4,dew0=16.4,dew1=19.2,dew2=17.2,dew3=19.2,h-eff=610.4,t-eff=26.0,hum-gain=34.74,CR0-1=-0.20,CR0-2=-0.20,CR2-1=0.00,CR2-2=0.20,end=1\r\n'
ERROR:root:Error, retrying with backoff
Traceback (most recent call last):
  File "/home/christoph/arduino-influxdb/collect.py", line 70, in ReadLoop
    float(timestamp)
ValueError: could not convert string to float: 'elapsed=60011i,t0=17.90,t1=19.20,t2=22.89,t3=22.20,rh0=91.20,ah0=13.9,rh1=100.00,ah1=16.5,rh2=70.20,ah2=14.3,rh3=83.30,ah3=16.4,dew0=16.4,dew1=19.2,dew2=17.2,dew3=19.2,h-eff=610.4,t-eff=26.0,hum-gain=34.74,CR0-1=-0.20,CR0-2=-0.20,CR2-1=0.00,CR2-2=0.20,end=1'
^CTraceback (most recent call last):
  File "/home/christoph/arduino-influxdb/collect.py", line 215, in <module>
    main()
  File "/home/christoph/arduino-influxdb/collect.py", line 211, in main
    writer.join()
  File "/usr/lib/python3.10/threading.py", line 1096, in join
    self._wait_for_tstate_lock()
  File "/usr/lib/python3.10/threading.py", line 1116, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
KeyboardInterrupt

I have included the last succeeding line for comparison.

If you take a closer look at the second (i.e. the failing) received line, you can see that it is cut off mid-way:

DEBUG:root:Received line b'FTX_log ventstate=1,ledventstate=1,buttonpressed=0i,ledventstatechanged=0i,brightness=25i,desiredventstate=1,error0=0i,error1=0i,error2=0i,error3=0i,error4=0i,error5=0i,lederror=0i,printerror=0i,timerup=20464546i,timerdown=0i,timerhigh=0i,old_t_vals0=0i,old_t_vals1=0i,old_t_vals2=0i,old_t_vals3=0i,old_h_vals0=0i,old_h_vals1=0i,old_h_vals2=0i,old_h_vals3=0i,sen

And after that, a new line is received, starting with FTX...

So, for some reason, the transmission of that line failed and we shall for now not care why. What I'm trying to understand is whether there is a way how the script can be made to handle (i.e. dismiss) such incomplete lines.

Is there a way?

As I write, it occurs to me that I might have set the timeout too high (I set it to 130 seconds, while the arduino is sending every 60 seconds), so if the line is dismissed if no end of line is received after the timeout, then that will save the problem. Will try it now. But let me know if there is another/better way.