openworm
Create unit test that verifies correct I/V curve for ca_boyle NML2 channel
We currently have a process to run the iv_analyse.py script on the output of the analyse_ca_boyle.sh script. This produces a figure.
We'd like to augment this process with an automated test that compares the output of iv_analyse with Figure 3A from the paper.
A good candidate to be done with a sciunit test @rgerkin , although anyone who can do this even in a simpler way would make a contribution.
Related to #20. @VahidGh and @rayner this would be the logical next step on the work you have both been doing related to I/V curves.
The unit test should then be instrumented to run on Travis CI in this repo.
@slarson, Great, I was looking at the sciunit+neuronunit documentation and samples, today. This issue also could be a good starting point for instrumenting the ion channel modeling pipeline with both sciunit+Travis CI for the verification & validation step. I don't know if validating single ion channel via neuronunit needs a test design with training phase (as suggested in this paper) or there is some predesignated test available in existing libraries!
I've submitted some pull requests to get some of the required tools into a better position for integration. These allow the analyse*.sh scripts to be decomposed into python functions, including NML2ChannelAnalyse and jnml.
I've also begun work on a neuronunit module to make use of these tools to run all the tests from within python without any command line invocations.
Once that is in order I can create NeuronUnit tests that recapitulate the analysis parameters themselves. As for testing/training as @VahidGh suggested, I think it depends on whether we are looking to test how good our models are, or whether we are trying to improve them by free parameter fitting. For now I think it would be good to know how close we are with the pieces we currently have, and then we can invest effort into model fitting.
Some of what I've worked on is here. I've updated neuronunit's jnml module (which will eventually be merged into pyNeuroML to do most of what we need to in Python, although I am still relying on jNeuroML over pyLEMS for model compatibility reasons that need to be worked out.
I can reproduce the I-V curve, so in principle I am ready to devise the validation test itself, but I wanted to point out some weird behavior in the simulation in the first few time steps. See cell Out[240] near the bottom of the notebook linked above. There is some sort of oscillation of the current in the first few steps, and I'm wondering if that is causing problems with the measurement of the peak current, and whether those oscillations should even be there. Maybe it's a time step size issue.
Pasting the image in question here for simplicity:
Hmm yeah that oscillation is strange. @rayner we haven't seen this right? Does it change if you change the time step?
@VahidGh On this issue, can you remind what that digitiser software is that you found for transforming the raster I/V plot curves from papers into numbers? Can you post an example of its output? Apologies if you have already posted this somewhere and I've lost track of it.
This is an online version, and this is a stand-alone one.
As an example, this is the digitized version for V=15 mv:
And this is the published Plotly version of the WebPlotDigitizer output.
@rgerkin I'd like to talk about using SciUnit (and NeuronUnit) when making validation test templates for different parts of this project. Is there any chance we can have a chat about this? Thanks.
This is worth looking back into again now that we have made progress in comparing simulated and observed I/V curves in ChannelWorm.
@slarson Do you have the DOI for that paper in your first message? I want to see if we have the Figure 3A data in the ion_channel db in ChannelWorm.
@slarson Do you know where that Fig. 3 came from in your first message? That's what I need to move forward. I thought maybe it came from this referenced in #20 but that's a different paper. I tried Google reverse image searching it but sadly that doesn't work yet for most journal figures (probably you screenshotted part of a pdf).
@rgerkin, the fig. 3A is taken from Boyle & Cohen 2008 paper. Which originally comes from Fig. 2B of this study which I've addressed how to access to data in the ipython file.