Is bxb evaluation suspended by shutdown for test annotations?

[chesnokov-yuriy]

Dear George

Looking up shutdown annotation behaviour at Obtaining Test Annotation Files:

Shutdown annotations. If the device suspends its analysis because of poor signal quality or for any other reason, it should mark the periods during which analysis is suspended. The evaluation software tallies beats missed during such periods separately from beats missed at other times.

I had impression that missing test beat annotations suspended by shutdown are excluded from comparison with reference.

However running bxb and peeking at its code I was unable to confirm or refute my conjecture.

Sincerely Yuriy

$ bxb -r 118e00 -a atr qtr -f 5:0 -t 6:0
bxb: (warning) nonstandard comparison selected
Beat-by-beat comparison results for record 118e00
Reference annotator: atr
     Test annotator: qtr

               Algorithm
        n    v    f    q    o    x
   _______________________________
 N |   68    0    0    0    0    6
 V |    0    0    0    0    0    0
 F |    0    0    0    0    0    0
 Q |    0    0    0    0    0    0
 O |    0    0    0    0
 X |    0    0    0    0

           QRS sensitivity:  91.89% (68/74)
 QRS positive predictivity: 100.00% (68/68)
           VEB sensitivity:      -  (0/0)
 VEB positive predictivity:      -  (0/0)
   VEB false positive rate:  0.000% (0/68)


  Beats missed in shutdown:   8.11% (6/74)
      N missed in shutdown:   8.11% (6/74)
      V missed in shutdown:      -  (0/0)
      F missed in shutdown:      -  (0/0)
       Total shutdown time:     5 seconds

with qtr annotation be exact copy of atr modified by adding shutdown annotations and writing back with wrann the following change:

...
    5:09.069   111265     R    0    0    0
    5:09.711   111496     A    0    0    0
    5:10.000   111600     ~   -1    0    0
    5:15.000   113400     ~    0    0    0
    5:15.564   113603     R    0    0    0
    5:16.378   113896     R    0    0    0
...

Is QRS sensitivity expected to be 100% instead and beats missed in shutdown as is 8.11% to be counted as quoted above:

evaluation software tallies beats missed during such periods separately from beats missed at other times.

What happens if no beats are removed from shutdown period e.g. noisy artifacts are detected instead the output of bxb is completely different as though without suspended by shutdown evaluation

bxb: (warning) nonstandard comparison selected
Beat-by-beat comparison results for record 118e00
Reference annotator: atr
     Test annotator: qtr

               Algorithm
        n    v    f    q    o    x
   _______________________________
 N |   70    0    0    0    4    0
 V |    0    0    0    0    0    0
 F |    0    0    0    0    0    0
 Q |    0    0    0    0    0    0
 O |    0    0    0    0
 X |    0    0    0    0

           QRS sensitivity:  94.59% (70/74)
 QRS positive predictivity: 100.00% (70/70)
           VEB sensitivity:      -  (0/0)
 VEB positive predictivity:      -  (0/0)
   VEB false positive rate:  0.000% (0/70)


  Beats missed in shutdown:   0.00% (0/74)
      N missed in shutdown:   0.00% (0/74)
      V missed in shutdown:      -  (0/0)
      F missed in shutdown:      -  (0/0)
       Total shutdown time:     1 seconds

with qtr annotation be exact copy of atr modified by adding shutdown annotations and writing back with wrann the following change:

...
    5:09.069   111265     R    0    0    0
    5:09.711   111496     A    0    0    0
    5:10.000   111600     ~   -1    0    0
    5:10.619   111823     R    0    0    0
    5:11.456   112124     |    0    0    0
    5:12.292   112425     R    0    0    0
    5:13.103   112717     |    0    0    0
    5:13.911   113008     |    0    0    0
    5:14.758   113313     |    0    0    0
    5:15.000   113400     ~    0    0    0
    5:15.564   113603     R    0    0    0
    5:16.378   113896     R    0    0    0
...