tedana icon indicating copy to clipboard operation
tedana copied to clipboard

Output RMSE map and time series for decay model fit

Open tsalo opened this issue 1 year ago • 1 comments

Closes #861. @madisoth, what kind of outputs do you think would be useful? I was thinking (1) a mean RMSE map and (2) maybe a mean RMSE time series, so users could flag volumes with bad model fits?

I could probably convert the RMSE map to a more meaningful unit, like percent?

Changes proposed in this pull request:

  • Add a function to calculate RMSE 4D time series from monoexponential model fit.

tsalo avatar Feb 21 '24 16:02 tsalo

Codecov Report

Attention: Patch coverage is 94.25287% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 89.91%. Comparing base (af5e99a) to head (730354a).

Files Patch % Lines
tedana/decay.py 93.33% 1 Missing and 1 partial :warning:
tedana/io.py 84.61% 1 Missing and 1 partial :warning:
tedana/workflows/tedana.py 80.00% 0 Missing and 1 partial :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1044      +/-   ##
==========================================
+ Coverage   89.80%   89.91%   +0.11%     
==========================================
  Files          26       26              
  Lines        3540     3621      +81     
  Branches      621      629       +8     
==========================================
+ Hits         3179     3256      +77     
- Misses        212      214       +2     
- Partials      149      151       +2     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Feb 21 '24 16:02 codecov[bot]

@madisoth gave me the thumbs up on this offline, so I'm opening it up for review.

tsalo avatar Mar 26 '24 15:03 tsalo

Here's the mean RMSE image for the 3-echo dataset:

Screenshot 2024-04-12 at 12 09 19 PM

Here's the timeseries plot (with +/- 1 SD) for the same dataset:

Screenshot 2024-04-12 at 12 09 30 PM

tsalo avatar Apr 12 '24 16:04 tsalo

The results are now substantially different:

Screenshot 2024-04-22 at 9 33 58 AM

tsalo avatar Apr 22 '24 15:04 tsalo

The results are now substantially different:

I do expect the values to be non-trivially lower because you were including voxels with 1 good echo, which meant fit coefficients of 0 (i.e. the residual error would be the entire time series).

handwerkerd avatar Apr 22 '24 21:04 handwerkerd

Ah, okay. That's a relief then.

tsalo avatar Apr 22 '24 22:04 tsalo

That's fixed in my most recent commit.

tsalo avatar Apr 26 '24 17:04 tsalo

@handwerkerd I think everything should be good now. Would you mind taking another look?

tsalo avatar Apr 26 '24 18:04 tsalo

Thanks @handwerkerd!

Here's what the updated 3-echo timeseries plot looks like, for posterity.

Screenshot 2024-04-27 at 9 33 58 AM

tsalo avatar Apr 27 '24 13:04 tsalo