validation analyses

[jeremymanning]

do these separately for synthetic data and real (fMRI) data:

• approximate coverage and "electrode" distribution to match ECoG data (pyFR)
• approximate variability across subjects's models (in synthetic data) to match ECoG (or fMRI) data
• how well can we:
  • recover models
  • predict electrode timeseries (correlations)
  • for fMRI data, what does the map look like (of how well we can reconstruct, by location)

[jeremymanning]

For ECoG data, it'd be useful to know how much of the data is needed to reach a stable estimate of the correlation model:

• for each subject, subsample out a contiguous block of x% of the recording session
• estimate the correlation model using that block
• compare (correlate) with correlation matrix estimated using full session
• then, across subjects, compute the average correlation (between sub-sampled and "full" correlation matrices) as a function of x

[jeremymanning]

We could attempt to evaluate the per-location information by comparing within-subject correlation matrices with and without each electrode

[jeremymanning]

Expanding on the previous comment in this issue, here's the specific analysis I had in mind:

• For each subject, compute the correlation matrix for all of their electrodes (let's call this C)
• Now, for each electrode i in turn:
  • Compute C_(not i): the correlation matrix between all of the non-i electrodes
  • Blur C_(not i) out to include location i. Let's call this expanded-out matrix C_i.
  • Compute the correlation between row i of C vs. C_i
  • Keep track of these correlations across all electrodes (and also keep track of where those electrodes are)
• Now make a plot similar to Fig. 5 from our preprint-- correlation by location. This will tell us: by location, how much might we expect the answers we get out to change if the electrodes at that location are vs. aren't included. In other words, if the correlations are high, that means that the electrode is non-essential, since we'd get the same answer via spatial blur. But if the correlations are low, those are regions to implant electrodes into, since the answers change drastically when that electrode is included in the analysis.

[lucywowen]

1. a) Subsampling subject and electrodes from pyfr data and correlating subsampled model to full model
2. b) Subsampling subject and electrodes from pyfr data, predicting, and comparing the correlations to prediction from full model
3. Subsample fMRI data (intact condition of pieman data) at corresponding pyfr locations, create a model from this data, and predict every where else