neurolib icon indicating copy to clipboard operation
neurolib copied to clipboard

Testing: test_models with dynamic loading

Open caglorithm opened this issue 2 years ago • 1 comments

Right now, we have a list of all models to test and have a function for each of them. This is kind of ugly.

Therefore, I think we should have some method of dynamically getting a list of all models and importing them programmatically, and subsequently, testing them one by one.

Some code that can be useful for this:

import glob
import importlib
import sys
import inspect

models = [p.split("/")[-1] for p in glob.glob("../neurolib/models/*") if not (p.split("/")[-1].startswith("_") or p.split("/")[-1].endswith(".py"))]
models.sort()
for model in models:
    i = importlib.import_module(f".models.{model}", "neurolib")
    classname = inspect.getmembers(i)[0][0]
    importname = f"neurolib.models.{model}"
    desc = inspect.getdoc(inspect.getmembers(i)[0][1]).split("\n")[0]
    print(f"Model: {model}\tClass: {classname}\n\t\tDescr: {desc}")
    print(f"\t\tImport: from {importname} import {classname}")

Outputs:

Model: aln	Class: ALNModel
		Descr: Multi-population mean-field model with exciatory and inhibitory neurons per population.
		Import: from neurolib.models.aln import ALNModel
Model: bold	Class: BOLDModel
		Descr: Balloon-Windkessel BOLD simulator class.
		Import: from neurolib.models.bold import BOLDModel
Model: fhn	Class: FHNModel
		Descr: Fitz-Hugh Nagumo oscillator.
		Import: from neurolib.models.fhn import FHNModel
Model: hopf	Class: HopfModel
		Descr: Stuart-Landau model with Hopf bifurcation.
		Import: from neurolib.models.hopf import HopfModel
Model: multimodel	Class: ALNNetwork
		Descr: Whole brain network of adaptive exponential integrate-and-fire mean-field
		Import: from neurolib.models.multimodel import ALNNetwork
Model: thalamus	Class: ThalamicMassModel
		Descr: Two population thalamic model
		Import: from neurolib.models.thalamus import ThalamicMassModel
Model: wc	Class: WCModel
		Descr: The two-population Wilson-Cowan model
		Import: from neurolib.models.wc import WCModel
Model: ww	Class: WWModel
		Descr: Wong-Wang model. Original version and reduced version.
		Import: from neurolib.models.ww import WWModel

caglorithm avatar Jun 20 '22 13:06 caglorithm

Btw, personally, I do not think this is necessary. If we want to simplify testing, I would rather do something like this:

class NativeTestCase(unittest.TestCase):
    model = None

    def test_single_node(self):
        # this will skip running test for this parent class
        if self.model is None:
            return
        self.model.run()
        ...
    def test_network(self):
        if self.model is None:
            return
        self.model.run()
        ...

class TestAln(NativeTestCase):
    model = ALNModel()

class TestHopf(NativeTestCase):
    model = HopfModel()

    def test_specific(self):
        # optional specific test for this model
        return 42
        

...

Yes, we still need to add tests for each (new) model manually, but all the logic is in one class, so adding a test is a matter of two lines... The same logic can be done for MultiModel.

jajcayn avatar Feb 20 '23 09:02 jajcayn