nest-simulator icon indicating copy to clipboard operation
nest-simulator copied to clipboard

Failing assertion upon `import nest`

Open jougs opened this issue 2 years ago • 4 comments

The following issue was reported on the mailing list:

python: /home/neurobit/src/nest-simulator/nestkernel/model_manager.h:321: size_t nest::ModelManager::get_num_connection_models() const: Assertion `connection_models_[ 0 ].size() <= invalid_synindex' failed.
[dcccluster:979935] *** Process received signal ***
[dcccluster:979935] Signal: Aborted (6)
[dcccluster:979935] Signal code:  (-6)
[dcccluster:979935] [ 0] /usr/lib64/libpthread.so.0(+0x12c20)[0x7f8c3a3cdc20]
[dcccluster:979935] [ 1] /usr/lib64/libc.so.6(gsignal+0x10f)[0x7f8c398a337f]
[dcccluster:979935] [ 2] /usr/lib64/libc.so.6(abort+0x127)[0x7f8c3988ddb5]
[dcccluster:979935] [ 3] /usr/lib64/libc.so.6(+0x21c89)[0x7f8c3988dc89]
[dcccluster:979935] [ 4] /usr/lib64/libc.so.6(+0x2fa76)[0x7f8c3989ba76]
[dcccluster:979935] [ 5] /home/neurobit/local/nest_3.3/lib64/python3.8/site-packages/nest/../../../nest/libnestkernel.so.3(+0x1aad22)[0x7f8c2a878d22]
[dcccluster:979935] [ 6] /home/neurobit/local/nest_3.3/lib64/python3.8/site-packages/nest/../../../nest/libnestkernel.so.3(_ZN4nest12ModelManager26register_connection_model_EPNS_14ConnectorModelE+0x32d)[0x7f8c2a80f07d]
[dcccluster:979935] [ 7] /home/neurobit/local/nest_3.3/lib64/python3.8/site-packages/nest/../../../nest/libmodels.so.3(_ZN4nest12ModelManager25register_connection_modelINS_24vogels_sprekeler_synapseEEEvRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_28RegisterConnectionModelFlagsE+0x171)[0x7f8c2b04d071]
[dcccluster:979935] [ 8] /home/neurobit/local/nest_3.3/lib64/python3.8/site-packages/nest/../../../nest/libmodels.so.3(_ZN4nest12ModelsModule4initEP14SLIInterpreter+0x2c14)[0x7f8c2afa0404]
[dcccluster:979935] [ 9] /home/neurobit/local/nest_3.3/lib64/python3.8/site-packages/nest/../../../nest/libsli.so.3(_ZN9SLIModule7installERSoP14SLIInterpreter+0x59)[0x7f8c29a22369]
[dcccluster:979935] [10] /home/neurobit/local/nest_3.3/lib64/python3.8/site-packages/nest/../../../nest/libsli.so.3(_ZN14SLIInterpreter9addmoduleEP9SLIModule+0x44)[0x7f8c299c29f4]
[dcccluster:979935] [11] /home/neurobit/local/nest_3.3/lib64/python3.8/site-packages/nest/../../../nest/libnest.so.3(_Z11neststartupPiPPPcR14SLIInterpreterNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x13cc)[0x7f8c2b6d37ec]
[dcccluster:979935] [12] /home/neurobit/local/nest_3.3/lib64/python3.8/site-packages/nest/pynestkernel.so(+0x42475)[0x7f8c2b920475]
[dcccluster:979935] [13] /usr/lib64/libpython3.8.so.1.0(+0x1d8faf)[0x7f8c3a9dcfaf]
[dcccluster:979935] [14] /usr/lib64/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x976)[0x7f8c3a9e5406]
[dcccluster:979935] [15] /usr/lib64/libpython3.8.so.1.0(_PyFunction_Vectorcall+0x1bf)[0x7f8c3a9a37af]
[dcccluster:979935] [16] /usr/lib64/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x5da)[0x7f8c3a9e506a]
[dcccluster:979935] [17] /usr/lib64/libpython3.8.so.1.0(_PyEval_EvalCodeWithName+0x3e7)[0x7f8c3a9a23c7]
[dcccluster:979935] [18] /usr/lib64/libpython3.8.so.1.0(PyEval_EvalCode+0x23)[0x7f8c3a9a35a3]
[dcccluster:979935] [19] /usr/lib64/libpython3.8.so.1.0(+0x25114b)[0x7f8c3aa5514b]
[dcccluster:979935] [20] /usr/lib64/libpython3.8.so.1.0(+0x1667e0)[0x7f8c3a96a7e0]
[dcccluster:979935] [21] /usr/lib64/libpython3.8.so.1.0(PyVectorcall_Call+0x70)[0x7f8c3a96f390]
[dcccluster:979935] [22] /usr/lib64/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x68d9)[0x7f8c3a9eb369]
[dcccluster:979935] [23] /usr/lib64/libpython3.8.so.1.0(_PyEval_EvalCodeWithName+0x3e7)[0x7f8c3a9a23c7]
[dcccluster:979935] [24] /usr/lib64/libpython3.8.so.1.0(_PyFunction_Vectorcall+0x422)[0x7f8c3a9a3a12]
[dcccluster:979935] [25] /usr/lib64/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x54d9)[0x7f8c3a9e9f69]
[dcccluster:979935] [26] /usr/lib64/libpython3.8.so.1.0(_PyFunction_Vectorcall+0x1bf)[0x7f8c3a9a37af]
[dcccluster:979935] [27] /usr/lib64/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x976)[0x7f8c3a9e5406]
[dcccluster:979935] [28] /usr/lib64/libpython3.8.so.1.0(_PyFunction_Vectorcall+0x1bf)[0x7f8c3a9a37af]
[dcccluster:979935] [29] /usr/lib64/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x5da)[0x7f8c3a9e506a]
[dcccluster:979935] *** End of error message ***
Aborted (`core' generado)
  • Rocky Linux 8.5
  • NEST 3.3 (Installation from source)
  • Problem also exists with current git

jougs avatar Jul 04 '22 09:07 jougs

Could you try running nest directly as follows:

source <nest_install_dir>/bin/nest_vars.sh
nest

and see if that works? If that does not work, could you try

sli -c "statusdict info" 
sli -c "statusdict /architecture get info"

and report the output? Could you also look at libnestutil/config.h in the build directory and post these lines corresponding to these (towards the end of the file):

/* Bit counts for Target members */
#define TARGET_BITS_SPLIT 0
#define TARGET_BITS_SPLIT_STANDARD 0
#define TARGET_BITS_SPLIT_HPC 1

heplesser avatar Jul 04 '22 10:07 heplesser

Executing the "source ..." and "nest" command shows the same error:

nest: /home/neurobit/src/nest_3.3_src/nestkernel/model_manager.h:321: size_t nest::ModelManager::get_num_connection_models() const: Assertion `connection_models_[ 0 ].size() <= invalid_synindex' failed.
[dcccluster:1177271] *** Process received signal ***
[dcccluster:1177271] Signal: Aborted (6)
[dcccluster:1177271] Signal code:  (-6)
[dcccluster:1177271] [ 0] /lib64/libpthread.so.0(+0x12c20)[0x7f34fe352c20]
[dcccluster:1177271] [ 1] /lib64/libc.so.6(gsignal+0x10f)[0x7f34fd63837f]
[dcccluster:1177271] [ 2] /lib64/libc.so.6(abort+0x127)[0x7f34fd622db5]
[dcccluster:1177271] [ 3] /lib64/libc.so.6(+0x21c89)[0x7f34fd622c89]
[dcccluster:1177271] [ 4] /lib64/libc.so.6(+0x2fa76)[0x7f34fd630a76]
[dcccluster:1177271] [ 5] /home/neurobit/local/nest_3.3/bin/../lib64/nest/libnestkernel.so(+0x187722)[0x7f34ffddc722]
[dcccluster:1177271] [ 6] /home/neurobit/local/nest_3.3/bin/../lib64/nest/libnestkernel.so(_ZN4nest12ModelManager26register_connection_model_EPNS_14ConnectorModelE+0x32d)[0x7f34ffd720ed]
[dcccluster:1177271] [ 7] /home/neurobit/local/nest_3.3/bin/../lib64/nest/libmodels.so(_ZN4nest12ModelManager25register_connection_modelINS_24vogels_sprekeler_synapseEEEvRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_28RegisterConnectionModelFlagsE+0x171)[0x7f35005a93b1]
[dcccluster:1177271] [ 8] /home/neurobit/local/nest_3.3/bin/../lib64/nest/libmodels.so(_ZN4nest12ModelsModule4initEP14SLIInterpreter+0x2c14)[0x7f35004fc7e4]
[dcccluster:1177271] [ 9] /home/neurobit/local/nest_3.3/bin/../lib64/nest/libsli.so(_ZN9SLIModule7installERSoP14SLIInterpreter+0x59)[0x7f34ff1ba149]
[dcccluster:1177271] [10] /home/neurobit/local/nest_3.3/bin/../lib64/nest/libsli.so(_ZN14SLIInterpreter9addmoduleEP9SLIModule+0x44)[0x7f34ff15a7d4]
[dcccluster:1177271] [11] nest[0x4119a0]
[dcccluster:1177271] [12] nest[0x41020d]
[dcccluster:1177271] [13] /lib64/libc.so.6(__libc_start_main+0xf3)[0x7f34fd624493]
[dcccluster:1177271] [14] nest[0x41053e]
[dcccluster:1177271] *** End of error message ***
Aborted (`core' generado)

Executing sli -c "statusdict info" shows:

Jul 05 15:40:48 CopyFile [Error]:
    Could not open source file.
Error in nest resource file: /BadIO in CopyFile_
--------------------------------------------------
Name                     Type                Value
--------------------------------------------------
architecture             dictionarytype      <dictionarytype>
argv                     arraytype           <arraytype>
built                    stringtype          Jun 23 2022 21:46:14
environment              dictionarytype      <dictionarytype>
exitcode                 integertype         0
exitcodes                dictionarytype      <dictionarytype>
files                    arraytype           <arraytype>
have_gsl                 booltype            true
have_libneurosim         booltype            false
have_mpi                 booltype            true
have_music               booltype            true
have_sionlib             booltype            true
host                     stringtype          x86_64-pc-linux
hostcpu                  stringtype          x86_64
hostos                   stringtype          linux
hostvendor               stringtype          pc
interactive              booltype            false
is_mpi                   booltype            false
ndebug                   booltype            false
platform                 stringtype          default
prefix                   stringtype          /home/neurobit/local/nest_3.3
prgdatadir               stringtype          /home/neurobit/local/nest_3.3/share/nest
prgdocdir                stringtype          /home/neurobit/local/nest_3.3/share/doc/nest
threading                stringtype          openmp
userargs                 arraytype           <arraytype>
version                  stringtype          3.3
--------------------------------------------------
Total number of entries: 26

Executing sli -c "statusdict /architecture get info" shows:

--------------------------------------------------
Name                     Type                Value
--------------------------------------------------
ArrayDatum               integertype         48
Dictionary               integertype         56
DictionaryDatum          integertype         56
double                   integertype         8
have_long_long           booltype            true
int                      integertype         4
IntegerDatum             integertype         40
long                     integertype         8
long_long                integertype         8
Token                    integertype         16
TokenArray               integertype         16
TokenArrayObj            integertype         40
TokenMap                 integertype         48
void_ptr                 integertype         8
--------------------------------------------------
Total number of entries: 14

The last lines of the libnestutil/config.h file are

/* Bit counts for Target members */
#define TARGET_BITS_SPLIT 1
#define TARGET_BITS_SPLIT_STANDARD 0
#define TARGET_BITS_SPLIT_HPC 1

/* Whether to enable detailed NEST internal timers */
/* #undef TIMER_DETAILED */

#endif // #ifndef CONFIG_H

xavierotazuGDS avatar Jul 05 '22 13:07 xavierotazuGDS

Did you configure NEST with the CMake option -Dtarget-bits-split=hpc? That likely explains the effect. Could you configure and build NEST again without this option and try again?

With hpc, NEST only uses 6 bits to encode up to 64 synapse model IDs, but NEST by default comes with 74 synapse models (because most synapse models come in plain, _lbl and _hpc variants). In the standard configuration, 9 bits are available for up to 512 synapse model IDs. The hpc setting should only be required in extreme cases (>= 2^18 MPI processes or more than 512 threads per process).

We clearly need to handle this situation better, but I'd appreciate your feedback on whether my suggestion fixes the problem first.

heplesser avatar Jul 05 '22 20:07 heplesser

You are right! Initially I compiled NEST with -Dtarget-bits-split=hpc

Now I have deactivated this option, compiled and reinstalled NEST. Now it works correctly. I can "import nest" from python and no error appear.

xavierotazuGDS avatar Jul 07 '22 14:07 xavierotazuGDS

Issue automatically marked stale!

github-actions[bot] avatar Sep 06 '22 08:09 github-actions[bot]