jep
jep copied to clipboard
Sometimes got error SIGSEGV when using numpy
@RequestMapping("/**")
public synchronized Object exec(@RequestBody String content, HttpServletRequest request) {
String requestUri = request.getRequestURI();
String jsonParam = JSONObject.parseObject(content).toJSONString();
PythonScript script = execution.getPythonScript(requestUri);
if(script!=null) {
try (Jep jep = new SharedInterpreter()) {
String model = extractModel(script.getPath());
String modelImportCmd = "from " + model + " import "+ script.getFunc();
jep.exec(modelImportCmd);
Object result = jep.invoke(script.getFunc(), jsonParam);
return result;
} catch (Exception e) {
throw new RuntimeException(e);
}
} else {
throw new RuntimeException("The requestUri("+requestUri+") has not been config the Python script in the path: " + execution.getScriptRoot());
}
}
Sometimes the code runs OK, but sometimes got the following errors and the program stoped:
- ERROR 1:
*** Aborted at 1627296117 (unix time) try "date -d @1627296117" if you are using GNU date *** PC: @ 0x0 (unknown) *** SIGSEGV (@0x2b77d8087e00) received by PID 15991 (TID 0x2b78aa92c700) from PID 18446744073039019520; stack trace: *** @ 0x2b77d87dd6d0 (unknown) @ 0x2b77d9cf8b6b ciBytecodeStream::get_method() @ 0x2b77d9c263e3 GraphBuilder::invoke() @ 0x2b77d9c22ebf GraphBuilder::iterate_bytecodes_for_block() @ 0x2b77d9c2487f GraphBuilder::iterate_all_blocks() @ 0x2b77d9c24e0a GraphBuilder::GraphBuilder() @ 0x2b77d9c2ab6a IRScope::IRScope() @ 0x2b77d9c2b6f9 IR::IR() @ 0x2b77d9c0d277 Compilation::build_hir() @ 0x2b77d9c0f3f0 Compilation::compile_java_method() @ 0x2b77d9c0f59a Compilation::compile_method() @ 0x2b77d9c0fa10 Compilation::Compilation() @ 0x2b77d9c10198 Compiler::compile_method() @ 0x2b77d9d6cc9c CompileBroker::invoke_compiler_on_method() @ 0x2b77d9d6e8d8 CompileBroker::compiler_thread_loop() @ 0x2b77da366bbb JavaThread::thread_main_inner() @ 0x2b77da366ec1 JavaThread::run() @ 0x2b77da1f5132 java_start() @ 0x2b77d87d5e25 start_thread @ 0x2b77d8f04bad __clone @ 0x0 (unknown)
- ERROR 2:
*** Aborted at 1627350873 (unix time) try "date -d @1627350873" if you are using GNU date *** PC: @ 0x0 (unknown) *** SIGSEGV (@0x2ad282005080) received by PID 31929 (TID 0x2ad342335700) from PID 18446744071595643008; stack trace: *** @ 0x2ad2826546d0 (unknown) @ 0x2ad28402b333 Monitor::wait() @ 0x2ad283bdcab2 CompileQueue::get() @ 0x2ad283be538b CompileBroker::compiler_thread_loop() @ 0x2ad2841ddbbb JavaThread::thread_main_inner() @ 0x2ad2841ddec1 JavaThread::run() @ 0x2ad28406c132 java_start() @ 0x2ad28264ce25 start_thread @ 0x2ad282d7bbad __clone @ 0x0 (unknown)
Does the hs_err_pid list what library it crashed in? libjvm.so or something else? And what JRE are you using?
In hs_err_pid it shows the following which I think is your want:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00002b5db2a93738, pid=20770, tid=0x00002b5d67f27700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_291-b10) (build 1.8.0_291-b10)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.291-b10 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [_multiarray_umath.cpython-36m-x86_64-linux-gnu.so+0x14c738] PyArray_Item_INCREF+0xf8
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
The crash is in the multiarray library, which is part of numpy. I see you're already using SharedInterpreter which is the best advice I can give you for using numpy with Jep. Unfortunately numpy wasn't written with sub-interpreters in mind and may not work correctly even when using SharedInterpreter. There's not really much Jep can do about this.
If you're going to try and solve it, I would try to narrow down what lines of Python are causing the crash and see if you can crash it with a normal CPython interpreter (not Jep).