long filenames error when using compcor

[davidreydellet]

Hello,

I used spmup applied to BIDS data with 2 different sets of options. It works well with the 1st set and I may have identified a bug when using compcor as in my 2nd set of options.

I get this warning:

Warning: 'sub_d1s01_ses_lsd_task_rest_acq_ep2d_space_MNI152NLin2009_desc_preprocessed_bold_design' exceeds MATLAB's maximum name length of 63 characters and has been truncated to 'sub_d1s01_ses_lsd_task_rest_acq_ep2d_space_MNI152NLin2009_desc_'._ 

Followed by these errors:

Error using fileparts (line 37)
Input must be a row vector of characters or string scalar.

Error in spmup_BIDS_1rstlevel (line 256)
            [filepath,filename,ext]=fileparts(matlabbatch{1}.spm.stats.fmri_spec.sess(frun).scans);

Error in run_spmup_bids (line 169)
                    subject_sess = spmup_BIDS_1rstlevel(subject_sess, opt(s));

I tried to shorten the filenames to see whether the issue really comes from this but I did not manage to get under this 63 characters cap.

Here is a mock dataset to reproduce this error (with two scripts: one with a set of options that works fine, and one with a set of options yielding this error): /users/davidreydellet/mock_datasets/

[Remi-Gau]

hey @davidreydellet was there supposed to be a link to some data and code ?

[Remi-Gau]

Not 100% sure the warning and the errors are related.

Can you tell us what function / line the warning is happening at?

It seems to be more related to trying to create a variable or structure fieldname.

>> sub_d1s01_ses_lsd_task_rest_acq_ep2d_space_MNI152NLin2009_desc_preprocessed_bold_design = 1
Warning: 'sub_d1s01_ses_lsd_task_rest_acq_ep2d_space_MNI152NLin2009_desc_preprocessed_bold_design'
exceeds the MATLAB maximum name length of 63 characters and will be truncated to
'sub_d1s01_ses_lsd_task_rest_acq_ep2d_space_MNI152NLin2009_desc_'. 

sub_d1s01_ses_lsd_task_rest_acq_ep2d_space_MNI152NLin2009_desc_ =

     1

>> some_structure.sub_d1s01_ses_lsd_task_rest_acq_ep2d_space_MNI152NLin2009_desc_preprocessed_bold_design = 1
Warning: 'sub_d1s01_ses_lsd_task_rest_acq_ep2d_space_MNI152NLin2009_desc_preprocessed_bold_design'
exceeds the MATLAB maximum name length of 63 characters and will be truncated to
'sub_d1s01_ses_lsd_task_rest_acq_ep2d_space_MNI152NLin2009_desc_'. 

some_structure = 

  struct with fields:

    sub_d1s01_ses_lsd_task_rest_acq_ep2d_space_MNI152NLin2009_desc_: 1

[Remi-Gau]

MATLAB is pretty strict as to what can be a valid variable or field name: can be an interoperability issue when creating tsv headers or JSON fieldnames that may work in python but lead to "fun" things with matlab.

https://bids-standard.github.io/bids-starter-kit/folders_and_files/metadata.html#interoperability-issues

[davidreydellet]

Hello Remi-Gau,

Thanks for your answers. I am actually a bit unsure how to share this mock dataset but here is the script:

% SPMup
BIDS_dir                = '/users/davidreydellet/mock_datasets/raw/'; % specify project folder
options                 = spmup_getoptions(BIDS_dir);
subject_list            = {sub-d1s01};

options.outdir          = '/users/davidreydellet/mock_datasets/derivative/';
options.Ncores          = 4;
options.subjects	= subject_list;
options.anat            = {'_T1w','_T2w'};
options.task 		= {'rest'};
options.norm_res	= 3; % normalized BOLD images will be 3 mm isotropic
options.motionexp	= 'on';
options.nuisance	= 'compcor';
options.carpet_plot	= 'on';

[BIDS,subjects]         = spmup_BIDS_unpack(BIDS_dir,options); % unpacks

[subjects,opt]          = run_spmup_bids(BIDS,subjects,options); % that's the magic bit

I am working on Matlab, and to my knowledge my json files have been encoded/decoded using matlab only. Regarding the BIDS format, I check with BIDS validator and it seems like my folder is BIDS-compliant.

The unpacking part works fine, the warning and the error happen when the function run_spmup_bids is running.

[CPernet]

i notice this warning for file name too long .. working on fixing that at some point , although it haven't failed for me? just warned

[CPernet]

so error is unrelated to the warning - you can see the issue is when parsing the scan name @davidreydellet are you ok with a bit of coding? you could put a debug point in https://github.com/CPernet/spmup/blob/master/bids/run_spmup_bids.m#L168 which means the processing is done then check subject_sess it should contains the right scan names in, but likely not hence the bug ; then report to us and we'll see what we can do to help you from there

thx for testing the tool

[davidreydellet]

Thank you for developing this tool and following its implementation!

I checked a bit, at this stage (run_spmup_bids L168), the variable subject_sess does contain filenames but I presume some are missing. In my case, my raw data comes with T1w and T2w images, fieldmaps, and a functional data file. I will deep dive a bit into spmup_BIDS_1rstlevel.

subject_sess.mat.tar.gz

[davidreydellet]

So actually, line 256 of spmup_BIDS_1rstlevel:

[filepath,filename,ext]=fileparts(matlabbatch{1}.spm.stats.fmri_spec.sess(frun).scans);

matlabbatch{1}.spm.stats.fmri_spec.sess(frun).scans is a cell, which doesn't fit the correct format for the function fileparts.

It seems to work if I modify it with char(matlabbatch{1}.spm.stats.fmri_spec.sess(frun).scans)

I modified my local version of spmup and I have launched it with compcor on a real dataset. I will update you if it worked.

[CPernet]

ok but it works in other BIDS datasets we tried, so we need to figure out why in your case, as this is not a tenable solution ; let me check again with the file your shared

[davidreydellet]

The first 7 subjects of my dataset got processed with this modification but an error appeared at subject#7. I will try to sort out this error now.

------------------------------
 running spmup_timeseries on /data1/lsd-basel/data/derivative/dataset1/sub-d1s07/ses-lsd/func/run1/sub-d1s07_ses-lsd_task-rest_acq-ep2d_space-MNI152NLin2009_desc-preprocessed-GLMdenoised_bold.nii
------------------------------
Checking Yeo's networks to organize GM voxels
Error using corr (line 91)
Requires a data matrix X.

Error in spmup_timeseriesplot (line 297)
    tmp                              = corr(tmp);

Error in spmup_BIDS_1rstlevel (line 263)
        spmup_timeseriesplot(VfMRI(1).fname, ...

Error in run_spmup_bids (line 169)
                    subject_sess = spmup_BIDS_1rstlevel(subject_sess, opt(s));

[CPernet]

x likely empty??

[davidreydellet]

Indeed, in this loop going through all ROIs, when it comes to roi 5, tmp is filled with NaNs and subsequently becomes 0, hence the error with the corr function. From what I understood, roi 5 is the limbic network in Yeo's 7 networks atlas. I guess it could be that this region in particular is absent from my functional data. I visually checked the raw functional data, it seems ok. Nonetheless, when it comes to the derivative functional data, it is poorly registered in the MNI space, hence the error to identify voxels in the limbic network I guess. I noticed this error with another subject too.

In all cases, this seems like an error that comes from my data itself rather than the code so I don't want to mix things up in this thread.

To come back to the initial error, the char(matlabbatch{1}.spm.stats.fmri_spec.sess(frun).scans) implementation works for me but I agree it's rather peculiar that spmup worked on bids datasets without this modification.

[CPernet]

@davidreydellet can you pull the updated version and see if setting options.reorient = 'off'; solves some of the problem @Remi-Gau options.reorient = 'off'; goes around issue https://github.com/CPernet/spmup/issues/5 while ultimately we need a better code to reorient for defaced data