Segmentation Fault when using Kratos with mmg

[heySourabh]

Description Kratos produces a segmentation fault during the transfer of mmg mesh

Scope

• MeshingApplication

To Reproduce

• Install mmg
• create virtual python environment: /home/user/anaconda3/bin/python -m venv /home/user/kratos_env/
• activate the environment: cd /home/user/kratos_env/ source bin/activate
• pip install Kratos: pip install KratosMultiphysics-all
• cd to example using mmg: cd /home/user/kratos_examples/mmg_remeshing_examples/use_cases/beam2D/source
• run Kratos: python MainKratos.py

It works fine for few steps (also produces output) with initial mesh but seg faults after "step_frequency" iterations which is 10 by default.

---

I have also tried compiling from source (cloned from GitHub) and after adding the following applications:

add_app ${KRATOS_APP_DIR}/LinearSolversApplication
add_app ${KRATOS_APP_DIR}/StructuralMechanicsApplication
add_app ${KRATOS_APP_DIR}/FluidDynamicsApplication
add_app ${KRATOS_APP_DIR}/MeshMovingApplication
add_app ${KRATOS_APP_DIR}/FSIApplication
add_app ${KRATOS_APP_DIR}/MappingApplication
add_app ${KRATOS_APP_DIR}/MeshingApplication

and, I have built mmg from source (with shared libraries) and added to configure.h

-DINCLUDE_MMG=ON                                                    \
-DMMG_ROOT="/home/user/mmg/build/"                                  \

It compiles without any errors. When running with mmg application however I get segmentation fault same as in the case of pip-install. Other applications seem to work fine. Also the mmg applications from the validation folder, hessian2D works fine.

Here are the last few lines of the output:

::[KSM Simulation]:: : STEP:  8 
::[KSM Simulation]:: : TIME:  0.08 
::[KSM Simulation]:: : Adaptative strategy converged in  1 iterations 
::[KSM Simulation]:: : STEP:  9 
::[KSM Simulation]:: : TIME:  0.09 
::[KSM Simulation]:: : Adaptative strategy converged in  1 iterations 
::[KSM Simulation]:: : STEP:  10 
::[KSM Simulation]:: : TIME:  0.09999999999999999 
::[KSM Simulation]:: : Adaptative strategy converged in  1 iterations 
::[KSM Simulation]:: : STEP:  11 
::[KSM Simulation]:: : TIME:  0.10999999999999999 
MMG Remeshing Process: Calculating the metrics 
MMG Remeshing Process: Remeshing 
MMG Remeshing Process: Remesh finished 
Segmentation fault

Expected behavior Should not result in segmentation fault.

Environment

• OS: Ubuntu 22.04
• Branch: master
• Python 3.9.12
• I have tried with mmg current GitHub version and also older mmg v5.5.0

Please let me know if you need any further information. Some other examples in the mmg_remeshing_examples/use_cases/ also produce Segmentation fault.

Thank you! Sourabh Bhat

[loumalouomega]

Which are the compilation flag of your mmg?, recently we changed the suggested configure, as we recommended some compilation flags that may not be compatible with old CPU

[loumalouomega]

I don't know if we are packing in pypi the MMG

[loumalouomega]

See https://github.com/KratosMultiphysics/Kratos/blob/master/applications/MeshingApplication/custom_external_libraries/mmg/build/configure.sh

[loumalouomega]

If not this looks like the memory is not properly cleaned

[heySourabh]

Thank you @loumalouomega for the quick reply. I will compile Kratos and mmg again from scratch, with the new mmg build script. I will report my observations in some time.

[heySourabh]

It still results in a segmentation fault. Here are the steps I followed.

1. Clone MMG: git clone https://github.com/MmgTools/mmg.git mmg
2. Run the a MMG build script (configure.sh) provided by you:

#!/bin/bash

# Set compiler
export CC=${CC:-gcc}
export CXX=${CXX:-g++}

# Configure
cmake                                                        \
-DCMAKE_BUILD_TYPE=Release                                   \
-DCMAKE_C_COMPILER=${CC}                                     \
-DCMAKE_CXX_COMPILER=${CXX}                                  \
-DCMAKE_C_FLAGS="-O3 -march=native -mtune=native -fopenmp"   \
-DCMAKE_CXX_FLAGS="-O3 -march=native -mtune=native -fopenmp" \
-DUSE_SCOTCH=OFF                                             \
-DLIBMMG_SHARED=ON                                           \
-DLIBMMG_STATIC=OFF                                          \
-DLIBMMGS_SHARED=ON                                          \
-DLIBMMGS_STATIC=OFF                                         \
-DLIBMMG2D_SHARED=ON                                         \
-DLIBMMG2D_STATIC=OFF                                        \
-DLIBMMG3D_SHARED=ON                                         \
-DLIBMMG3D_STATIC=OFF                                        \

# Build
cmake --build $(pwd) -- -j$(nproc)

The build worked without any errors.

---

3. Cloned Kratos: git clone https://github.com/KratosMultiphysics/Kratos.git Kratos

4. Copied standard_configure.sh as configure.sh

5. Added following applications to Kratos scripts/configure.sh

add_app ${KRATOS_APP_DIR}/LinearSolversApplication
add_app ${KRATOS_APP_DIR}/StructuralMechanicsApplication
add_app ${KRATOS_APP_DIR}/FluidDynamicsApplication
add_app ${KRATOS_APP_DIR}/MeshMovingApplication
add_app ${KRATOS_APP_DIR}/FSIApplication
add_app ${KRATOS_APP_DIR}/MappingApplication
add_app ${KRATOS_APP_DIR}/MeshingApplication

6. Following flags for Kratos compilation:

-DUSE_MPI=OFF                                                       \
-DUSE_EIGEN_MKL=OFF                                                 \
-DKRATOS_GENERATE_PYTHON_STUBS=ON                                   \
-DINCLUDE_MMG=ON                                                    \
-DMMG_ROOT="/home/user/Downloads/mmg_latest/mmg/"                   \

7. Compiled Kratos: cd Kratos/scripts/ sh configure.sh

It compiled without any errors.

---

8. Created runner.sh

KRATOS_HOME=/home/user/Downloads/Kratos
export PYTHONPATH=$PYTHONPATH:$KRATOS_HOME/bin/Release
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$KRATOS_HOME/bin/Release/libs:$KRATOS_HOME/bin/Release
$KRATOS_HOME/bin/Release/runkratos $*

9. cd to the error causing example folder (downloaded from ): cd /home/user/Downloads/Examples-master/mmg_remeshing_examples/use_cases/beam2D/source/

10. Run Kratos: /home/user/Downloads/Kratos/bin/Release/runner.sh MainKratos.py

Unfortunately, following the above steps results in a Segmentation fault for the mesh adaptation examples using MMG. Other applications work fine. Can you spot any mistake in the steps above?

[loumalouomega]

Did you try to run the tests instead of the example?, maybe the example is old and must be updated

[heySourabh]

What is the correct way to run tests? I tried: cd Kratos/applications/MeshingApplication/tests home/user/Downloads/Kratos/bin/Release/runner.sh test_MeshingApplication.py

where runner.sh is the script I created in the point 8 in my previous comment.

I get the following warnings & errors: ** VTK library not found. Unavailable file format. -> multiple times

Running python tests ... 
[WARNING] Unittests: TetrahedraReconnectUtility process is not compiled and the corresponding tests will not be executed

test_isosurface_remesh_sphere (test_remesh_sphere.TestRemeshMMG3D) ...   ** VTK library not founded. Unavailable file format.
  ** VTK library not founded. Unavailable file format.
  ** VTK library not founded. Unavailable file format.
  ** VTK library not founded. Unavailable file format.

  ## Warning: MMG3D_saveSol: no metric data to save.
7.161s ...ERROR

test_remesh_sphere (test_remesh_sphere.TestRemeshMMG3D) ...   ** VTK library not founded. Unavailable file format.
  ** VTK library not founded. Unavailable file format.
  ** VTK library not founded. Unavailable file format.
  ** VTK library not founded. Unavailable file format.
0.710s ...FAIL
test_remesh_sphere_skin (test_remesh_sphere.TestRemeshMMG3D) ...   ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
0.286s ...FAIL
test_remesh_sphere_skin_prisms (test_remesh_sphere.TestRemeshMMG3D) ...   ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
  ## WARNING: WRONG SOLUTION NUMBER. IGNORED
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
[WARNING] Geometry: Computation of local coordinates failed at iteration 0
[WARNING] Geometry: Computation of local coordinates failed at iteration 0
:
[WARNING] Geometry: Computation of local coordinates failed at iteration 0
[WARNING] Geometry: Computation of local coordinates failed at iteration 0
0.441s ...ok

test_execution (meshing_application_test_factory.TwoDDynamicBeamTest) ... [WARNING] DEPRECATION-Warning; MechanicalSolver: Using "line_search", please move it to "solving_strategy_settings" as "type" 
[WARNING] DEPRECATION-WARNING: The input strategy string is not in lower case letters. Please change it from "Hessian" to "hessian" 
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
0.213s ...ok

test_execution (meshing_application_test_factory.TwoDDynamicBeamTest) ... [WARNING] DEPRECATION-Warning; MechanicalSolver: Using "line_search", please move it to "solving_strategy_settings" as "type" 
[WARNING] DEPRECATION-WARNING: The input strategy string is not in lower case letters. Please change it from "Hessian" to "hessian" 
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
0.213s ...ok
test_execution (meshing_application_test_factory.TwoDDynamicBeamLineLoadTest) ... [WARNING] DEPRECATION-Warning; MechanicalSolver: Using "line_search", please move it to "solving_strategy_settings" as "type" 
[WARNING] DEPRECATION-WARNING: The input strategy string is not in lower case letters. Please change it from "Hessian" to "hessian" 
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
[WARNING] InternalVariablesInterpolationProcess: WARNING:: INTERPOLATION TYPE NOT AVALAIBLE OR EMPTY LIST
0.173s ...ok
test_execution (meshing_application_test_factory.ThreeDShellTest) ... [WARNING] DEPRECATION-Warning; MechanicalSolver: Using "line_search", please move it to "solving_strategy_settings" as "type" 
[WARNING] DEPRECATION-WARNING: The input strategy string is not in lower case letters. Please change it from "Hessian" to "hessian" 
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
  ** VTK library not found. Unavailable file format.
0.179s ...ok
test_execution (meshing_application_test_factory.ThreeDDynamicBeamTest) ... [WARNING] DEPRECATION-Warning; MechanicalSolver: Using "line_search", please move it to "solving_strategy_settings" as "type" 
[WARNING] DEPRECATION-WARNING: The input strategy string is not in lower case letters. Please change it from "Hessian" to "hessian" 
  ** VTK library not founded. Unavailable file format.
  ** VTK library not founded. Unavailable file format.
  ** VTK library not founded. Unavailable file format.
  ** VTK library not founded. Unavailable file format.
1.888s ...ok

======================================================================
ERROR: test_isosurface_remesh_sphere (test_remesh_sphere.TestRemeshMMG3D)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/bsourabh/Downloads/Kratos/applications/MeshingApplication/tests/test_remesh_sphere.py", line 483, in test_isosurface_remesh_sphere
    check_files.ExecuteFinalize()
  File "/home/bsourabh/Downloads/Kratos/bin/Release/KratosMultiphysics/compare_two_files_check_process.py", line 96, in ExecuteFinalize
    self.__CompareSolMetricFile()
  File "/home/bsourabh/Downloads/Kratos/bin/Release/KratosMultiphysics/compare_two_files_check_process.py", line 348, in __CompareSolMetricFile
    lines_ref, lines_out = self.__GetFileLines()
  File "/home/bsourabh/Downloads/Kratos/bin/Release/KratosMultiphysics/compare_two_files_check_process.py", line 128, in __GetFileLines
    raise Exception(err_msg)
Exception: The specified output file name "/home/bsourabh/Downloads/Kratos/applications/MeshingApplication/tests/mmg_eulerian_test/test_sphere_isosurface_step=0.o.sol" is not valid!

======================================================================
FAIL: test_remesh_sphere (test_remesh_sphere.TestRemeshMMG3D)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/bsourabh/Downloads/Kratos/applications/MeshingApplication/tests/test_remesh_sphere.py", line 145, in test_remesh_sphere
    check.ExecuteFinalizeSolutionStep()
  File "/home/bsourabh/Downloads/Kratos/bin/Release/KratosMultiphysics/from_json_check_result_process.py", line 51, in ExecuteFinalizeSolutionStep
    self.assertTrue(self.process.IsCorrectResult(), "Results do not coincide with the JSON reference\n" + self.process.GetErrorMessage())
AssertionError: False is not true : Results do not coincide with the JSON reference

followed by many more lines like:

Error checking for variable DISTANCE, results:
Node 173 0.08639867!=0.09437596; rel_tol=1e-06, abs_tol=0.001

----------------------------------------------------------------------
Ran 9 tests in 11.466s

FAILED (failures=2, errors=1)

[marcnunezc]

Can you try disabling nodal interpolation?

In the json:

https://github.com/KratosMultiphysics/Examples/blob/2f5e7e70158b6a7bf7a3da47197764c34896e5f7/mmg_remeshing_examples/use_cases/beam2D/source/ProjectParameters.json#L90

Add the following parameters:

"interpolate_nodal_values"         : false,
"interpolate_non_historical"       : false,

Since it happens at runtime and just after remeshing this is the first thing I thought of.

If this is the case we will try to find a fix.

[heySourabh]

Thanks @marcnunezc for the suggestion. With the changes suggested here are the last few lines of output:

::[KSM Simulation]:: : Adaptative strategy converged in  1 iterations 
::[KSM Simulation]:: : STEP:  10 
::[KSM Simulation]:: : TIME:  0.09999999999999999 
::[KSM Simulation]:: : Adaptative strategy converged in  1 iterations 
::[KSM Simulation]:: : STEP:  11 
::[KSM Simulation]:: : TIME:  0.10999999999999999 
MMG Remeshing Process: Calculating the metrics 
MMG Remeshing Process: Remeshing 
MMG Remeshing Process: Remesh finished 
[WARNING] ResidualBasedBlockBuilderAndSolver: ATTENTION! setting the RHS to zero!
/home/user/Downloads/Kratos/bin/Release/runner.sh: line 4: 24588 Segmentation fault      $KRATOS_HOME/bin/Release/runkratos $*

It still seg faults however, there is an additional line in the output which was not shown before. WARNING] ResidualBasedBlockBuilderAndSolver: ATTENTION! setting the RHS to zero!

[loumalouomega]

Can you run with set OMP_NUM_THREADS=1?

[heySourabh]

Hello @loumalouomega I have modified the runner script to:

export OMP_NUM_THREADS=1
KRATOS_HOME=/home/user/Downloads/Kratos
export PYTHONPATH=$PYTHONPATH:$KRATOS_HOME/bin/Release
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$KRATOS_HOME/bin/Release/libs:$KRATOS_HOME/bin/Release
$KRATOS_HOME/bin/Release/runkratos $*

But I still get the Segmentation fault. I see one of the warnings while running tests (see: ). Is this utility necessary for working with mmg?

[loumalouomega]

Huuum, this is strange I will research more

[marcnunezc]

Huuum, this is strange I will research more

Btw, if I am not mistaken, you updated the Examples to follow what is done in #4558 right? If this is the case I think we should revert back the examples as that PR will probably take even longer.

[loumalouomega]

Huuum, this is strange I will research more

Btw, if I am not mistaken, you updated the Examples to follow what is done in #4558 right? If this is the case I think we should revert back the examples as that PR will probably take even longer.

The segmentation is happening in the tests, there is something else...

[heySourabh]

Huuum, this is strange I will research more

Btw, if I am not mistaken, you updated the Examples to follow what is done in #4558 right? If this is the case I think we should revert back the examples as that PR will probably take even longer.

I am using the latest master branch of Kratos. I believe it must be merged already, right?

[loumalouomega]

Huuum, this is strange I will research more

Btw, if I am not mistaken, you updated the Examples to follow what is done in #4558 right? If this is the case I think we should revert back the examples as that PR will probably take even longer.

I am using the latest master branch of Kratos. I believe it must be merged already, right?

Nope, that's one of the biggest and longest discussion ever, with no consensus

[loumalouomega]

With the recent changes is this still happening?

[heySourabh]

I have recompiled Kratos from latest source and followed the installation process for the meshing application. However, some of the mesh adaptation examples are still causing segmentation fault, after reading the adapted mesh generated by MMG. I will look into the Kratos code and try to debug the problem in the next week.

[SeiDPierre]

I have the same issue with different validation test from mmg_remeshing_example/ This is the output for hessian2D:

Traceback (most recent call last):
  File "/home/clouzetpierre/Documents/workspace/Kratos/Examples/mmg_remeshing_examples/validation/hessian2D/source/test_hessian.py", line 102, in <module>
    vtk_io = KratosMultiphysics.VtkOutput(main_model_part, vtk_settings)
RuntimeError: Error: The item with name "output_frequency" is present in this Parameters but NOT in the default values
Hence Validation fails
Parameters being validated are : 
{
    "condition_data_value_variables": [],
    "element_data_value_variables": [],
    "file_format": "ascii",
    "gauss_point_variables_extrapolated_to_nodes": [],
    "model_part_name": "MainModelPart",
    "nodal_data_value_variables": [
        "NODAL_H",
        "METRIC_TENSOR_2D"
    ],
    "nodal_flags": [],
    "nodal_solution_step_data_variables": [
        "DISTANCE"
    ],
    "output_control_type": "step",
    "output_frequency": 1.0,
    "output_precision": 7,
    "output_sub_model_parts": false,
    "save_output_files_in_folder": false
}
Defaults against which the current parameters are validated are :
{
    "condition_data_value_variables": [],
    "condition_flags": [],
    "custom_name_postfix": "",
    "custom_name_prefix": "",
    "element_data_value_variables": [],
    "element_flags": [],
    "file_format": "binary",
    "gauss_point_variables_extrapolated_to_nodes": [],
    "gauss_point_variables_in_elements": [],
    "model_part_name": "PLEASE_SPECIFY_MODEL_PART_NAME",
    "nodal_data_value_variables": [],
    "nodal_flags": [],
    "nodal_solution_step_data_variables": [],
    "output_control_type": "step",
    "output_interval": 1.0,
    "output_path": "VTK_Output",
    "output_precision": 7,
    "output_sub_model_parts": false,
    "save_output_files_in_folder": true,
    "write_deformed_configuration": false,
    "write_ids": false
}




in kratos/sources/kratos_parameters.cpp:1281: void Parameters::ValidateDefaults(const Parameters&) const
   kratos/sources/kratos_parameters.cpp:1309: void Parameters::ValidateDefaults(const Parameters&) const
   kratos/sources/kratos_parameters.cpp:1259: void Parameters::ValidateAndAssignDefaults(const Parameters&)

[loumalouomega]

Luckily is an easy to solve issue, replace output frequency with output_interval

[SeiDPierre]

Oh! Thanks! the beam2D in uses_cases/ still prensents seg fault too

[loumalouomega]

O will correct them

[SeiDPierre]

Hi @marcnunezc and @loumalouomega , the seg fault problem is preventing me from working properly now. I was wondering what is the status on that thread? I have run some valgrind and gdb tests with the Examples-master/mmg_remeshing_examples/use_cases/beam2D/example. one openMP thread is used. I am only sharing an output with --show-all-kinds=false since the file is >60Mb,, Also, what can I do to help out with that issue? valgrind_simple.txt

[loumalouomega]

Sorry for the delay, I have been very bust lately, now I have some extra time, I will try to take a look

[loumalouomega]

Hi @marcnunezc and @loumalouomega , the seg fault problem is preventing me from working properly now. I was wondering what is the status on that thread? I have run some valgrind and gdb tests with the Examples-master/mmg_remeshing_examples/use_cases/beam2D/example. one openMP thread is used. I am only sharing an output with --show-all-kinds=false since the file is >60Mb,, Also, what can I do to help out with that issue? valgrind_simple.txt

For the moment my run is clean, which version of MMG are you using?

[SeiDPierre]

5.7.1

[loumalouomega]

5.7.1

I just downloaded last master, I will try again, and compile in FullDebug

[SeiDPierre]

Which one are you usualyy using?

[loumalouomega]

I was using 5.6, a bit outdated