mysql_perf_analyzer icon indicating copy to clipboard operation
mysql_perf_analyzer copied to clipboard

myperf won't start

Open imadsani opened this issue 10 years ago • 8 comments

myperf started absolutely fine day 1 but won't start anymore.

Error: There is insufficient memory for the Java Runtime Environment to continue. Native memory allocation (mmap) failed to map 2555904 bytes for committing reserved memory.

Server: CentOS 6.7 64-bit Xeon E5450 32GB RAM Percona 5.6 JRE version: (8.0_51-b16) (build ) Java VM: OpenJDK 64-Bit Server VM (25.51-b03 mixed mode linux-amd64 compressed oops)

--------------- T H R E A D ---------------

Current thread (0x0000029610008800): JavaThread "Unknown thread" [_thread_in_vm, id=8769, stack(0x000002961805d000,0x000002961815e000)]

Stack: [0x000002961805d000,0x000002961815e000], sp=0x000002961815c6e0, free space=1021k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)

imadsani avatar Oct 05 '15 15:10 imadsani

Check how much free memory you have (such as command top, or free -m, etc).

You can also modify start_myperf.sh to remove "-Xms128m" so that it has no minimum memory requirement. Still, without enough memory, no app will work smoothly.

xrao avatar Oct 05 '15 15:10 xrao

This is the result of free -m

free -m total used free shared buffers cached Mem: 32108 6502 25605 10 118 2479 -/+ buffers/cache: 3904 28203 Swap: 16095 0 16095

imadsani avatar Oct 05 '15 15:10 imadsani

Check if you can find jvm error log file, usually with name like hs_err_pidxxxx.log, which could have more detail. I usually run mysql_perf_analyzer with jvm option "-Xms128m -Xmx4g" in start_myperf.sh.

xrao avatar Oct 05 '15 16:10 xrao

Tried changing Xmx to 4g but still not joy: At the top it says: The system is out of physical RAM or swap space In 32 bit mode, the process size limit was hit

Ram seems fine, but why does it give me the 32-bit error. I'm running: java-1.8.0-openjdk.x86_64

imadsani avatar Oct 05 '15 16:10 imadsani

You might have multiple versions of Java installed. Log in as the user launching mysql_perf_analyzer, and type "which java" and "java -version" to check your path and versions.

You can also locate the JDK path to use JDK bin/java, not jre/java. Then replace "java" inside start_myperf.sh with the full path.

xrao avatar Oct 05 '15 16:10 xrao

launching the application as root which java /usr/bin/java

java -version OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000029ba07b0000, 2555904, 1) failed; error='Operation not permitted' (errno=1)

There is insufficient memory for the Java Runtime Environment to continue. Native memory allocation (mmap) failed to map 2555904 bytes for committing reserved memory. An error report file with more information is saved as: /root/hs_err_pid19333.log

adding the full path doesn't help

imadsani avatar Oct 05 '15 17:10 imadsani

I finally got it to work, had to disable MPROTECT on java

imadsani avatar Oct 05 '15 17:10 imadsani

Interesting and thanks for sharing. Once when I tried to figure out the failures to spawn mysql threads, I ever wrote a simple Java program to test what impacted Linux threads, and I was surprised that Linux reported memory issue back to my java test code when actually other resources were at shortage. For me, the impact resources were "ulimit -u", "/proc/sys/kernel/pid_max", and "/proc/sys/vm/max_map_count"

xrao avatar Oct 05 '15 17:10 xrao