Cleanup kernel status and configuration flags

[terhorstd]

Current state nest.GetKernelStatus() returns a mixture of run-time states, compile time configurations and options. In code this is reflected with different mechanisms and on different levels:

• cmake compile flags and options via -Dwith-..., which inconsistently are ON/OFF flags in most cases, but sometimes take a path argument,
• Compiler HAVE_… preprocessor macros that partially are transfered identically from the with-… cmake-flags, and sometimes set explicitly in header files.
• legacy SLI flags that where used dynamically provide kernel settings such as is_threaded (bool) in the SLI status dict.

Additionally, there is the external script nest-config that returns partially the same information, together with flags and library paths for the compiler and linker. This information is usually required by external tools compiling modules, such as NEST ML. Many of these flags however become meaningless when deploying pre-built binaries with APT or PIP, as all shown paths are only available on the build-system which is usually not available to the users. The nest-config is a bash-script template filled by cmake with

• NEST code information (NEST_VERSION),
• compiler definitions (CMAKE_CXX_COMPILER, CMAKE_CXX_COMPILER_VERSION, …),
• compile environment configurations on the build-system (ALL_INCLUDES, Python_EXECUTABLE, library paths, …), and
• deployment configurations on the build-system (CMAKE_INSTALL_DIR, CMAKE_INSTALL_DOCDIR, …)

The current state is that an incomplete subset of this information is available via nest.GetKernelStatus() and a different incomplete subset is available from the nest-config shell script.

Where to go

1. make a concise and comprehensive definition of variable classes, such that it is clear where to put newly introduced variables in future.
2. specify expected variable types to avoid #ifdef HAVE_MPI problems (note that currently cmake sets HAVE_MPI=OFF, so using the conventional ifdefis ineffective)
3. find a way to refactor the structure of information (technically, conceptually, backwards compatible (?), …)

To start the discussion I could imagine a split at least into three classes of variables. In a hypothetical Python representation available via the imported nest object:

• nest.features: built-in options and feature-flags regarding the neuroscientific and operational domains ("detailed_timers", "mpi", "conductance_models", "gap_junctions", …)
• nest.build: static technical compile-time information ("version=3.9.0", "compiler_flags=-O3", "prefix=/path/to/nest", "mpiexec=orterun", "libgsl=/usr/lib/...", "boost=/lib/libboost…")
• nest.status: dynamic run-time information of the current NEST instance ("time_simulate", "time_construct", "ranks", "nlocalproc", "rss_size", "spike_compression", …)

Additional context This discussion was triggered by recurring problems in packaging where paths from the config tend to "leak" from the build-system. This is especially relevant when install locations of NEST and/or libraries are not known beforehand, such as in Python environments, and when requiring to reproduce the compile-time environment on the user machine, as with NESTML.

Configuration is found at least in these files:

• nestkernel/config.h.in
• bin/nest-config.in
• CMakeLists.txt

[terhorstd]

In a discussion with @gtrensch we came across a couple of other points:

• Variables such as HAVE_UINT64_T are build-system dependent and need proper encapsulation in our code requiring maximum a single reference (ideally none at all). Once the code is built these typenames are of no further interest as there should be no influence on the binary. Different typenames may however hint at very different library or compiler environments.
• Variables such as HAVE_READLINE, HAVE_BOOST, … are also build-system definitions of library ecosystem. The runtime environment should provide those libraries. Ideally we implement a functionality to test if the dynamic load is actually available at run-time to raise appropriate errors.
• Variables such as TARGET_BITS_SPLIT, TIMER_DETAILED and CONFIG_TICS_PER_STEP define behavior and features and need proper read-only representation at run-time if those are compile-time decisions.

[heplesser]

@terhorstd The PYNEST-NG branch has already a modernized approach to providing configuration information, see

https://github.com/heplesser/nest-simulator/blob/57d63a1301b1fea7da44be04ddb38723c5eb165e/nestkernel/kernel_manager.cpp#L28

This can then nicely be used from tests

https://github.com/heplesser/nest-simulator/blob/57d63a1301b1fea7da44be04ddb38723c5eb165e/testsuite/pytests/conftest.py#L77-L79

and shell scripts

https://github.com/heplesser/nest-simulator/blob/57d63a1301b1fea7da44be04ddb38723c5eb165e/testsuite/do_tests.sh#L128-L131

Maybe this provides a starting point for further work?