BlueBrain
Can we split pyast.cpp into multiple files?
When we compile NMODL with non-gcc compiler (e.g. Intel, PGI), pyast.cpp takes huge time too compile.
I wonder if this could be split / printed into multiple CPP files.
@ohm314 : what do you think?
Related question : when we call sympy through one of the ODE solver, is nmodl python module used? I assume not.
Edit:
terminate called after throwing an instance of 'pybind11::error_already_set'
what(): ModuleNotFoundError: No module named 'nmodl'
So yes we do use!
I was profiling compilation time with -time:
time /gpfs/bbp.cscs.ch/tds/home/kumbhar/soft/install/linux-rhel7-x86_64/gcc-8.3.0/pgi-19.10-64jfse/linux86-64-llvm/19.10/bin/pgc++ -D_nmodl_EXPORTS -I/gpfs/bbp.cscs.ch/ssd/apps/hpc/jenkins/deploy/tools/2020-02-01/linux-rhel7-x86_64/gcc-8.3.0/flex-2.6.3-3fjqz4ot4m/include -I/gpfs/bbp.cscs.ch/tds/home/kumbhar/workarena/systems/tds/repos/bbp/coreneuron/external/nmodl -I/gpfs/bbp.cscs.ch/tds/home/kumbhar/workarena/systems/tds/repos/bbp/coreneuron/external/nmodl/src -I/gpfs/bbp.cscs.ch/tds/home/kumbhar/workarena/systems/tds/repos/bbp/coreneuron/external/nmodl/build_1/src -I/gpfs/bbp.cscs.ch/tds/home/kumbhar/workarena/systems/tds/repos/bbp/coreneuron/external/nmodl/ext -I/gpfs/bbp.cscs.ch/tds/home/kumbhar/workarena/systems/tds/repos/bbp/coreneuron/external/nmodl/ext/cli11/include -I/gpfs/bbp.cscs.ch/tds/home/kumbhar/workarena/systems/tds/repos/bbp/coreneuron/external/nmodl/ext/pybind11/include -I/gpfs/bbp.cscs.ch/ssd/apps/hpc/jenkins/deploy/external-libraries/2020-02-01/linux-rhel7-x86_64/gcc-8.3.0/python-3.7.4-tfxecymrkn/include/python3.7m -D__STDC_VERSION__=199901L -D__GCC_ATOMIC_TEST_AND_SET_TRUEVAL=1 -DJSON_SKIP_UNSUPPORTED_COMPILER_CHECK=1 --diag_suppress 611,997,82,111 --display_error_number -DEIGEN_DONT_VECTORIZE=1 -fPIC --c++11 -A -o CMakeFiles/_nmodl.dir/pyast.cpp.o -c /gpfs/bbp.cscs.ch/tds/home/kumbhar/workarena/systems/tds/repos/bbp/coreneuron/external/nmodl/build_1/src/pybind/pyast.cpp -time -v -O0
And:
Timing stats:
init 16 millisecs 0%
parser 244267 millisecs 38%
expand 38305 millisecs 5%
schedule 356330 millisecs 55%
assemble 482 millisecs 0%
Total time 639400 millisecs
PGCC/x86-64 Linux 19.10-0: compilation successful
....
/gpfs/bbp.cscs.ch/tds/home/kumbhar/soft/install/linux-rhel7-x86_64/gcc-8.3.0/pgi-19.10-64jfse/linux86-64-llvm/19.10/share/llvm/bin/llc /tmp/pgc++7g1fX8fn8I6X.ll -march=x86-64 -mcpu=native -O0 -fast-isel=0 -non-global-value-max-name-size=4294967295 -x86-cmov-converter=0 -relocation-model=pic -filetype=obj -o CMakeFiles/_nmodl.dir/pyast.cpp.o
Action(ReadTIFile(./pyast.ti))
Unlinking /tmp/pgc++lg1fHJjMXV0O.il
Unlinking /tmp/pgc++Jg1fP-1OlQQ2.s
Unlinking /tmp/pgc++7g1fX8fn8I6X.ll
Unlinking /tmp/pgc++tg1f5Iv-b5lU.llvm
real 13m21.603s
user 12m56.559s
sys 0m21.714s
So 13 minutes to compile that file :)
With all python dependencies, nmodl should be really a wheel!
cc: @iomaganaris
ast.hpp should also be split because most visitors only need a few Ast class definitions.
Definitely, the generated files are getting huge and compile time out of hand. We should generate those classes into multiple files. It's not entirely clear to me yet what's the best way to split them up.
