Duration of time slices of an AnalogSignal differs when using or not lazy loading

[kohlerca]

Hi All,

I am using cut_segment_by_epoch to extract trials from a Blackrock Microsystems dataset (i140703-001.ns6 and i140703-001.nev publicly available here).

When cutting a fixed duration after the times defined in an Event object, the first dimension of the resulting AnalogSignal objects varies across the slices by 1 sample. This is odd, as the signal is sampled at 30 KHz and the interval is an integer number in ms (i.e. 300*pq.ms), which should yield 9000 samples. Data was loaded using lazy=True.

Apparently, there is a bug when getting time slices using AnalogSignalProxy objects. In the code, AnalogSignal and AnalogSignalProxy use different algorithms to find the values of the start/end indexes in the data array:

1. The AnalogSignal.time_slice function finds the number of samples corresponding to the time delta in the slice, and finds the end index by adding it to the start index;

2. The AnalogSignalProxy.load function finds the indexes that corresponds to t_start and t_stop in the slice. Since the Segment in this dataset does not have t_start == 0, the truncation using int() may result in one less sample due to floating-point error when correcting the offset.

I used the code below to cut a single slice and compare the behavior when using or not lazy loading, and the slices do indeed differ by the last sample. The one with lazy=False is the expected output, with 300 ms duration.

import neo
import quantities as pq
from neo.utils import add_epoch, cut_segment_by_epoch

file = "i140703-001"
io = neo.BlackrockIO(file, nsx_to_load=6)

for lazy in True, False:

    print(f"Cutting with lazy={lazy}\n\n")

    block = io.read_block(lazy=lazy, load_waveforms=False)

    events = block.segments[0].events[0]

    if lazy:
        events = events.load()

    # As no annotations are present, use the index of one of the 'CUE-ON'
    # events and create an Event object with the epoch start time
    event_time = events[144]
    cut_event = neo.Event([event_time.magnitude] * event_time.units)

    epoch = add_epoch(block.segments[0], cut_event,
                      pre=0*pq.ms, post=300*pq.ms, attach_result=False)

    trial = cut_segment_by_epoch(block.segments[0], epoch, reset_time=True)[0]

    print(f"Shape: {trial.analogsignals[0].shape}\n")
    print(f"One channel:\n {trial.analogsignals[0][:, 1]}\n")
    print(f"Duration: {trial.analogsignals[0].duration}\n\n")

Output:

Cutting with lazy=True


Shape: (8999, 96)

One channel:
 [[88.75]
 [93.75]
 [86.75]
 ...
 [30.25]
 [23.25]
 [18.5 ]] uV

Duration: 0.29996666666666666 1/Hz


Cutting with lazy=False


Shape: (9000, 96)

One channel:
 [[88.75]
 [93.75]
 [86.75]
 ...
 [23.25]
 [18.5 ]
 [19.75]] uV

Duration: 0.3 1/Hz

Expected behavior: Time slices should not differ between the two modes of loading data.

Version details: Debian GNU/Linux 9.13 (stretch) Python 3.9.4 neo==0.9.0 quantities==0.12.4

Thank you, Cristiano

[JuliaSprenger]

This looks like a small, but important detail we should fix. @samuelgarcia Any ideas on this. Could we use the same code in both cases to determine t_start and t_stop?

[JuliaSprenger]

We should then also add a test to check for consistent border handling.

[JuliaSprenger]

Fixed by https://github.com/NeuralEnsemble/python-neo/pull/1298