Introduce support for reloading dynamic libraries

[sergiorg-hpc]

IMPORTANT This experimental change depends on further discussions on how to properly reload the state of NEURON (e.g., by having a reload() method that cleans-up the state to its default). We can begin some of these discussions on this Draft PR, if needed, or in a separate channel.

Description

Reloading a dynamic library in NEURON would currently load the original version that was previously loaded in memory. This is mostly due to the fact that the handle is never stored nor closed, making the shared library ref. count to be always greater than 0. The following information from the official documentation confirms our initial hypothesis:

       ...
       If the same shared object is opened again with dlopen(), the same
       object handle is returned.  The dynamic linker maintains
       reference counts for object handles, so a dynamically loaded
       shared object is not deallocated until dlclose() has been called
       on it as many times as dlopen() has succeeded on it.
       ...

As a consequence, attempting to update the mechanisms by recompiling the library externally would not have any visible impact, even if we manually ask NEURON to load the same library once again.

How does it work

The proposed change utilizes a static map to cache shared library handles with their correspondent given path. The idea is to retrieve the previous handle stored in the map and close it if the same library is asked to be loaded twice, forcing it to reload.

Ideally, the handle of the libraries would make more sense to be part of the NEURON state, somehow. But this must be part of the discussions when designing the reload() method.

How to Test

Using the HOC interpreter, one can easily test the support for reloading the dynamic library by using nrn_load_dll() and a custom .mod file:

> create soma
> soma insert MyChannel
> soma psection()
soma { nseg=1  L=100  Ra=35.4
        /*location 0 attached to cell 1*/
        /* First segment only */
        insert morphology { diam=500}
        insert capacitance { cm=1}
        insert MyChannel { g_MyChannel=0.1 e_MyChannel=-70}
}

If we now rename MyChannel to MySecondChannel to avoid conflicts (see limitations below) and recompile the same .mod file without closing the interpreter, the following error appears by using the current version of NEURON:

> nrn_load_dll("/path/to/library.so")
loading membrane mechanisms from /path/to/library.so
Additional mechanisms from files
 "mytest.mod"
nrniv: The user defined name already exists: MyChannel
 near line 5
 nrn_load_dll("/path/to/library.so")
                                           ^
        nrn_load_dll("/path/to/li...")

This is due to the fact that the new shared library is not effectively loaded, meaning that MySecondChannel does not exist. By including the proposed changes, this is the result:

> nrn_load_dll("/path/to/library.so")
> soma insert MySecondChannel
> soma psection()
soma { nseg=1  L=100  Ra=35.4
        /*location 0 attached to cell 1*/
        /* First segment only */
        insert morphology { diam=500}
        insert capacitance { cm=1}
        insert MyChannel { g_MyChannel=0.1 e_MyChannel=-70}
        insert MySecondChannel { g_MySecondChannel=0.2 e_MySecondChannel=-70}
}

Known Limitations

Right now and despite loading the same library with the changes, the name resolution of NEURON prevents conflicting mechanisms to load. In addition, in the previous example MyChannel still exists, despite that we reloaded the library that contained its definition, which is incorrect.

Nonetheless, the changes proposed here are required, in some way or another (e.g., closing the shared library handle from the global reload() method).

[codecov-commenter]

Codecov Report

Merging #1318 (0339f89) into master (d9462cc) will increase coverage by 0.00%. The diff coverage is 75.00%.

@@           Coverage Diff           @@
##           master    #1318   +/-   ##
=======================================
  Coverage   32.18%   32.18%           
=======================================
  Files         570      570           
  Lines      108684   108688    +4     
=======================================
+ Hits        34980    34983    +3     
- Misses      73704    73705    +1     

Impacted Files	Coverage Δ
src/nrnoc/init.cpp	70.45% <75.00%> (+0.04%)	:arrow_up:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update d9462cc...0339f89. Read the comment docs.

[pramodk]

Thank you @sergiorg-hpc for detailed report and draft! we will discuss with @nrnhines about possible implementation method/API for global state cleanup.

[sergiorg-hpc]

Thank you @sergiorg-hpc for detailed report and draft! we will discuss with @nrnhines about possible implementation method/API for global state cleanup.

My pleasure, glad to help. But don't thank me only, thank @jorblancoa as well for the hard work these days to understand this issue.

[pramodk]

@nrnhines: A short call will help @sergiorg-hpc & @jorblancoa to finalise this PR.

Reloading library issue has been understood and fix would be straightforward. What we would like to understand is how to cleanup some global variables (e.g. list f pre-registered mechanisms etc.) cc: @alexsavulescu

[pramodk]

@nrnhines: there is a desire to work on this task / PR during the hackathon as @anilbey also outlined this as a possible project in our slide deck.

Do you think you would have time to outline a bit or put together a scaffold for what needs to be done? From our last discussion, I remember that we need to clean various global data structures including mechanism tables. One idea was to also introduce something like h.reset() or h.clear() to trigger the cleaning of the previous state. (a simple approach to begin with).

[nrnhines]

I'm happy to help with this. I guess an initial context for me would be to have a test folder with mod1 and mod2 subfolders containing identical hh.mod files but with some extra differently named functions and some differences in the BREAKPOINT and DERIVATIVE blocks (eg. at a minimum just printf statements).

nrnivmodl has a limitation that it only creates a specifically named arch folder, e.g. x86_64, but no problem if nrnivmodl is run separately in the mod1 and mod2 subfolders. Then one can load the proper libnrnmech.so with the proper path.

On the python side, I'd just construct a model encapsulated in class Model(): for easy construction destruction

On the other hand, focusing on clearing a model, perhaps a python model loader/unloader in pure python could be written where the loader takes, for example, a modeldb model, as an arg. I imagine complete success means that any set of modeldb models, or existing tests, can be loaded, run, unloaded, along with repeats, from a single launch of nrniv or python.

[pramodk]

Thanks @nrnhines for the summary!

Just an additional clarification: when I mentioned scaffold or outline, I meant more like making list of code places and data structures that needs to be cleared for reloading new mechanism library. I think that’s where more internal insights from you would be really helpful!

[nrnhines]

nrn/src/nrnoc/init.cpp is a good place to observe how a mechanism gets registered. Most relevant array indices are the type of the mechanism. membfunc is the primordial example and other type indexed arrays are

   memb_func_size_ = 30;
    memb_list.reserve(memb_func_size_);
    memb_func = (Memb_func*) ecalloc(memb_func_size_, sizeof(Memb_func));
    pointsym = (Symbol**) ecalloc(memb_func_size_, sizeof(Symbol*));
    point_process = (Point_process**) ecalloc(memb_func_size_, sizeof(Point_pro$
    pnt_map = static_cast<char*>(ecalloc(memb_func_size_, sizeof(char)));
    memb_func[1].alloc = cab_alloc;
    nrn_pnt_template_ = (cTemplate**) ecalloc(memb_func_size_, sizeof(cTemplate$
    pnt_receive = (pnt_receive_t*) ecalloc(memb_func_size_, sizeof(pnt_receive_$
    pnt_receive_init = (pnt_receive_init_t*) ecalloc(memb_func_size_, sizeof(pn$
    pnt_receive_size = (short*) ecalloc(memb_func_size_, sizeof(short));
    nrn_is_artificial_ = (short*) ecalloc(memb_func_size_, sizeof(short));
    nrn_artcell_qindex_ = (short*) ecalloc(memb_func_size_, sizeof(short));
    nrn_prop_param_size_ = (int*) ecalloc(memb_func_size_, sizeof(int));
    nrn_prop_dparam_size_ = (int*) ecalloc(memb_func_size_, sizeof(int));
    nrn_dparam_ptr_start_ = (int*) ecalloc(memb_func_size_, sizeof(int));
    nrn_dparam_ptr_end_ = (int*) ecalloc(memb_func_size_, sizeof(int));
    memb_order_ = (short*) ecalloc(memb_func_size_, sizeof(short));
    bamech_ = (BAMech**) ecalloc(BEFORE_AFTER_SIZE, sizeof(BAMech*));
    nrn_mk_prop_pools(memb_func_size_);
    nrn_bbcore_write_ = (bbcore_write_t*) ecalloc(memb_func_size_, sizeof(bbcor$
    nrn_bbcore_read_ = (bbcore_write_t*) ecalloc(memb_func_size_, sizeof(bbcore$
    nrn_nmodl_text_ = (const char**) ecalloc(memb_func_size_, sizeof(const char$
    nrn_nmodl_filename_ = (const char**) ecalloc(memb_func_size_, sizeof(const $
    nrn_watch_allocate_ = (NrnWatchAllocateFunc_t*) ecalloc(memb_func_size_,

void nrn_register_mech_common( fills in most of that information and needs to be looked at in detail as it also sets up Symbol tables for all the names from the mod file that will be available to the user through the interpreter. And adds the mechanism name Symbol to the top level symbol table.

There are another half dozen or so places to look for mechanism specific data pertaining to threads and python and artificial cells. and point processes.