bookkeeper icon indicating copy to clipboard operation
bookkeeper copied to clipboard

[FLAKY TEST] :bookkeeper-server:test failed with SIGSEGV on CI

Open dlg99 opened this issue 3 years ago • 2 comments

BUG REPORT

Test failed

org.apache.bookkeeper.bookie.CheckpointOnNewLedgersTest

  Test testCheckpoint PASSED
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f6668000840, pid=8491, tid=0x00007f66661e0700
#
# JRE version: OpenJDK Runtime Environment (Zulu 8.60.0.21-CA-linux64) (8.0_322-b06) (build 1.8.0_322-b06)
# Java VM: OpenJDK 64-Bit Server VM (25.322-b06 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  0x00007f6668000840
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/runner/work/bookkeeper/bookkeeper/bookkeeper-server/hs_err_pid8491.log
#
# If you would like to submit a bug report, please visit:
#   http://www.azul.com/support/
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007efc84000840, pid=8605, tid=0x00007efc81fee700
#
# JRE version: OpenJDK Runtime Environment (Zulu 8.60.0.21-CA-linux64) (8.0_322-b06) (build 1.8.0_322-b06)
# Java VM: OpenJDK 64-Bit Server VM (25.322-b06 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  0x00007efc84000840
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/runner/work/bookkeeper/bookkeeper/bookkeeper-server/hs_err_pid8605.log
#
# If you would like to submit a bug report, please visit:
#   http://www.azul.com/support/
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Process 'Gradle Test Executor 20' finished with non-zero exit value 134
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at https://docs.gradle.org/7.3.3/userguide/java_testing.html#sec:test_execution
org.gradle.process.internal.ExecException: Process 'Gradle Test Executor 20' finished with non-zero exit value 134
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at https://docs.gradle.org/7.3.3/userguide/java_testing.html#sec:test_execution
	at org.gradle.api.internal.tasks.testing.worker.ForkingTestClassProcessor.stop(ForkingTestClassProcessor.java:183)
	at org.gradle.api.internal.tasks.testing.processors.RestartEveryNTestClassProcessor.endBatch(RestartEveryNTestClassProcessor.java:77)
	at org.gradle.api.internal.tasks.testing.processors.RestartEveryNTestClassProcessor.processTestClass(RestartEveryNTestClassProcessor.java:55)
	at sun.reflect.GeneratedMethodAccessor427.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.FailureHandlingDispatch.dispatch(FailureHandlingDispatch.java:30)
	at org.gradle.internal.dispatch.AsyncDispatch.dispatchMessages(AsyncDispatch.java:87)
	at org.gradle.internal.dispatch.AsyncDispatch.access$000(AsyncDispatch.java:36)
	at org.gradle.internal.dispatch.AsyncDispatch$1.run(AsyncDispatch.java:71)
	at org.gradle.internal.concurrent.InterruptibleRunnable.run(InterruptibleRunnable.java:42)
	at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:61)
	at java.lang.Thread.run(Thread.java:750)

org.apache.bookkeeper.bookie.CookieTest
...
FAILURE: Build failed with an exception.

* What went wrong:
> Task :bookkeeper-server:test FAILED
Execution failed for task ':bookkeeper-server:test'.

Expected behavior

repeatedly passing test

dlg99 avatar Feb 10 '22 01:02 dlg99

hard to say without looking at .../bookkeeper-server/hs_err_pid8605.log but there is a chance this one is the same as https://github.com/apache/bookkeeper/issues/3040

@nicoloboschi is it possible to pick up logs like hs_err_pid.*\.log (if exists) from any directory in case of the test failure on CI and package with other logs for review? Unfortunately RocksDB segfaults end up creating these in working dir which happened to be bookkeeper-server/

# An error report file with more information is saved as:
# /home/runner/work/bookkeeper/bookkeeper/bookkeeper-server/hs_err_pid8605.log

dlg99 avatar Feb 11 '22 22:02 dlg99

maybe you need something like this: https://github.com/apache/pulsar/blob/60ef5e983e5e8956bd0b602b5741bd6255c6258a/.github/workflows/ci-unit-broker-client-api.yaml#L86-L99

alternatively, quick and dirty, https://github.com/apache/bookkeeper/blob/1eae5e473b7775419760747b5d1f4dadd0c09052/.github/workflows/backward-compat-tests.yml#L54

nicoloboschi avatar Feb 11 '22 22:02 nicoloboschi

Now we switch to maven, I think we can closed it.

hezhangjian avatar May 04 '24 01:05 hezhangjian