Integrate hnn-core for running simulations

[blakecaldwell]

We have been internally discussing this WIP for a few months; time to put it up on github.

The hnn-core project is a more flexible way to run NEURON (Python) simulations than was originally coded in run.py. It allows users to create, modify, and read native Python objects representing the parameters, network construction, simulation settings, and simulation results. There has been significant effort put into documentation, testing, and best-practices as part of hnn-core.

Our plan is to remove certain sections of code in HNN and migrate them to hnn-core. This will begin with the code reading the parameter files, running the simulation, and saving spiking and dipole data. Then HNN will import the hnn_core module to read parameter files, running the simulation, saving dipoles, and saving spiking data. Lastly, code in HNN that has been replaced by code in hnn_core as part of this first step needs to be removed.

Code changes part of this integration will not change the GUI functionality. Ideally, the transition will be transparent.

Work remaining:

• [x] Updating existing param files that use -1 for an input time (see https://github.com/jonescompneurolab/hnn-core/issues/180)
• [x] Add 'spec_cmap' parameter to hnn-core (https://github.com/jonescompneurolab/hnn-core/pull/184)
• [x] Print simulation status with MPI (https://github.com/jonescompneurolab/hnn-core/issues/112)
• [x] Update HNN documentation to use pip install neuron and reference a tagged HNN release @blakecaldwell
• [x] Somatic voltages (depends on: https://github.com/jonescompneurolab/hnn-core/issues/181) @ntolley
• ~~Somatic currents (depends on: https://github.com/jonescompneurolab/hnn-core/issues/187) @ntolley~~
• [x] Print status messages from hnn-core to the simulation log window (part of #232) @blakecaldwell
• [x] Save rawdpl files (depends on: https://github.com/jonescompneurolab/hnn-core/issues/183) @kohl-carmen
• [x] Handle empty spike files (depends on: https://github.com/jonescompneurolab/hnn-core/issues/178) @rythorpe
• [x] Tonic inputs (depends on: https://github.com/jonescompneurolab/hnn-core/pull/209) @jasmainak
• [x] Integrate hnn-core required functionality into #232 @blakecaldwell
  • Almost complete. A few tasks left in #232
• [x] Verify trial seeding and results between HNN and hnn-core @rythorpe (depends on: https://github.com/jonescompneurolab/hnn-core/issues/113)
• [ ] Add toggle between old and updated calcium dynamics (https://github.com/jonescompneurolab/hnn-core/issues/110) @samikane
• [x] Resolve reliability issue with MPI simulations in hnn-core (https://github.com/jonescompneurolab/hnn-core/pull/285)
• [x] Run through tutorials with the new code to verify functionality (not formal tests)
  • [x] ERP tutorial
  • [x] Alpha/Beta tutorial
  • [x] Gamma tutorial
  • [x] Optimization tutorial @blakecaldwell
• [x] Merge #232

Things pushed to another PR:

• [ ] Add back tooltips to the dialogs (https://github.com/jonescompneurolab/hnn-core/issues/214)
• [ ] Set global variables through the GUI
• [ ] Convert optimization to scipy instead of nlopt

[blakecaldwell]

A large part of this is done in #232, but I will need some help to complete this. I do not plan on working on this after the end of the month. @jasmainak @rythorpe @ntolley feel free to take issues on this list. I will finish up #112 as indicated.

[jasmainak]

I am going to be somewhat busy until the mid of next month ... trying to resist touching code, but happy to review PRs!

[blakecaldwell]

@jasmainak @stephanie-r-jones I have done as much as possible for this in PR #232. The items on the checklist relate to functionality missing from hnn-core. Once they are added, minor changes will be needed in #232 to complete integration.

Two aspects may take significant time and probably should begin ASAP:

• Somatic current/voltage recording in hnn-core (hnn-core #181)
• Making make sure that the hnn-core simulations are giving the same or comparable results to old HNN simulations. Significant progress has already been made in checking dipole results in hnn-core. Multiple trials need to be validated (hnn-core #113) Checking the other forms of model output can also be done in hnn-core.

[blakecaldwell]

@rythorpe what do you think about result comparisons between hnn-core and HNN? Do the model results need to be checked for consistency?

[blakecaldwell]

Thanks for the work on somatic voltages @ntolley! I'll integrate that into #232 and check off some of my todo's there.

It looks like https://github.com/jonescompneurolab/hnn-core/issues/183 is almost ready. Thanks @kohl-carmen!

Certifying hnn-core results as the new standard for HNN might be the most time-consuming task remaining. I defer to @rythorpe for a strategy for that.

[rythorpe]

@rythorpe what do you think about result comparisons between hnn-core and HNN? Do the model results need to be checked for consistency?

Comparing the seeding behavior between hnn-gui and hnn-core is turning out to be a bit more complicated than I expected. As I mentioned here, the discrepancy is hard for me to diagnose particularly when it comes to adding rhythmic inputs. I'll try to give an update this weekend.

[blakecaldwell]

While #232 is merged, this issue can remain open. Besides testing and fixing any new bugs, there is one issue in hnn-core that seriously affects the stability of parallel simulations. So a resolution to https://github.com/jonescompneurolab/hnn-core/pull/285 is another prerequisite to making this a release