Computation of windowSize based on detrendCutoff in localDetrend

[danibene]

Hello! Using a version of the removeTrend function ported to Python, I noticed that frequencies below the detrendCutoff in removeTrend are not completely attenuated when using the local method: https://github.com/sappelhoff/pyprep/issues/125

I was wondering if this is the desired output, since it isn't clear to me why the window size is computed by dividing 1.5 by the cutoff frequency: https://github.com/VisLab/EEG-Clean-Tools/blob/9ac9ea0c21d44b57f9e9f93b62ca727c7b75c73e/PrepPipeline/utilities/removeTrend.m#L79

Also, I tried to test the original MATLAB code, but I wasn't able to debug it myself. Any tips there would be appreciated.

Script:

%% simulate signal
duration = 120;
frequency = [0.2, 0.4, 0.8, 1.6, 3.2];
amplitude = [1, 2, 3, 2, 1];
srate = 200;

nSamples = round(duration * srate);
period = 1 / srate;
seconds = (1:nSamples).*period;
data = zeros(1, nSamples);
for i=1:length(frequency)
    data = data + amplitude(i) * sin(2 * pi * frequency(i) * seconds);
end
%% format input
EEG.data = data;
EEG.srate = srate;
detrendIn = struct('detrendChannels', 1, 'detrendType', 'linear', ...
                    'detrendCutoff', 0.5, 'detrendStepSize', 0.02, ...
                    'detrendCommand', []);
[EEG, detrendOut] = removeTrend(EEG, detrendIn);

Error:

testRemoveTrend

Unrecognized function or variable 'change_row_to_column'.

Error in localDetrend (line 35)
data = change_row_to_column(data);

Error in removeTrend (line 83)
        localDetrend(EEG.data(detrendOut.detrendChannels, :)', ...

Error in testRemoveTrend (line 30)
[EEG, detrendOut] = removeTrend(EEG, detrendIn);

[VisLab]

A few comments (corrected):

Detrending of linear type is done with the chronux_2 runline command. The change_row_to_column is part of chronux, which should be downloaded as part of the plugin so I don't know why there was a problem unless you didn't add it to your path. In PREP it is treated as local so just installing as an EEGLAB plugin (or not installing as an EEGLAB plugin) might not add it to your path correctly.

See @vasileermicioi for correct answer.

[vasileermicioi]

@VisLab 1.5/detrendFreq formula is reducing the frequency by 33%, which means using this formula with a detrendFreq=1Hz will have the same effect as using 1/detrendFreq with a detrendFreq=0.67Hz

[VisLab]

@vasileermicioi: Thank you for pointing that out!

:-(( I looked at the defaults as set in getPrepDefaults.m to see what we were setting them to. I misread the default as signal.rate/2, but the default is 1, the first parameter in the specification structure.

The rules for detrendCutoff are: it must be positive, numeric, scalar, and have a value less than the Nyquist frequency.

    case 'detrend'
          defaults = struct( ...
            'detrendCutoff', ...
            getRules(1, {'numeric'}, ...
            {'positive', 'scalar', '<', signal.srate/2}, ...
            'Frequency cutoff for detrending or high pass filtering.'), ...
  );

The key observation is that you should set your detrendCutoff based on the frequency of the physical phenomenon you are detrending and 0.67 Hz is a reasonable detrending interval for EEG.

I think we probably should have called this parameter detrendCutoffHz to be clearer.

[danibene]

@VisLab Thank you for your help, it was an issue with the path & the example code runs now.

Would you agree with redefining windowSize based on 1.0/detrendFreq as suggested by @vasileermicioi ?

I.e., instead of: https://github.com/VisLab/EEG-Clean-Tools/blob/9ac9ea0c21d44b57f9e9f93b62ca727c7b75c73e/PrepPipeline/utilities/removeTrend.m#L79

windowSize = 1.0/detrendOut.detrendCutoff; 

We were wondering if we should make this change in our implementation and were interested in your input on whether that would be correct.

[VisLab]

The removeTrend uses the linear detrending when the else clause is taken. The windowSize is set to 1.5/detrendOut.detrendCutoff.

Since the default for detrendCutoff is 1, the default is 1.5 seconds. or 0.6667 Hz. I think this is a good default setting for EEG.

What I was trying to convey is that the name detrendCutoff is probably not the best name for this parameter, but the calculation is just fine. If we were going to change the windowSize to 1/detrendOut.detrendCutoff we would probably want to change the default detrendCutoff to 0.6667 instead of 1. Remember if you have a strong feeling that the detrending should be done using a different default cutoff, Prep will still work with a modified setting.

[danibene]

@VisLab Thank you for your response. To confirm I understood:

1. when detrendType = 'linear', detrendCutoff = 1 will filter with a cutoff frequency of 0.6667 Hz?
2. when detrendType = 'high pass', detrendCutoff = 1 will filter with a cutoff frequency of 1 Hz?

[VisLab]

That is correct. Remember these are different methods and they do different things to the signal and cutoff frequencies are not direct equivalents.

Most of our testing was done with the high pass method. However, based on my experience, I don't believe that the detection of bad channels and robust reference is very sensitive to this choice or to the exact choice of frequency as long as it is in the ballpark.

Prep makes a big effort to produce a final signal that is not filtered or otherwise detrended so that users are free to select their filtering/trending based on application after referencing.