trex-stateless-gui
trex-stateless-gui copied to clipboard
cisco-system-traffic-generator
Trex gui is consuming tons of cpu and memory after some time
I usually leave the gui running and after some time (5h+) the gui gets unusable and slows down the whole system. here is a ps dump after 2 days letting it running without any traffic generation:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
phschoen 18763 93.3 3.6 12493080 1139896 pts/42 SI+ Nov 16 3758:10 java -jar trex-stateless-gui.jar
Hi,
is the GUI connected to trex during all this period? Or there is only GUI window open and nothing else?
gui is opened and connected to a remote server which is not sending something. however some ports are reserved (2 out of 8). I will try if its also happening if no server is connected.
actually its even growing just by opening the trex gui and leave it running. Also each click in a drop down menu is increasing the memory e.g. opening and closing the help dropdown is increasing the memory by 40k each time
Hi,
regarding menubar memory leak, I've checked on very lightweight jfx app on Java 8 (which we are using), and the same leak is reproduced, I've just selected dropdown and hold right arrow, and memory started to grow. So this issue is related to jfx itself, maybe in future if we will migrate to newer java and javafx it will be resolved (if newer javafx doesn't have this issue).
regrading main memory leak seems that I've found an error (looks like it related to logs, because once I've disabled logging for some classes who spam to logs and connected to trex memory is stalled around 257 MiB.
Please let me know your platforms (linux/windows/mac) and I'll prepare a build for your so you can check on your scenario.
Thanks, Egor.
HI, @phschoen
Please check latest v4.5.5. Once I've validated it was around 305-307 maximal after 30-40 minutes.
hi, i tested the v4.5.5 on Ubuntu Linux and used java 8.211:
$java -version
java version "1.8.0_211"
Java(TM) SE Runtime Environment (build 1.8.0_211-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.211-b12, mixed mode)
memory seems fine but cpu usage still goes over the roof and slows the pc down after some time (1 core with ~95% use no mater what).
Please, let me know latest scenario, how did you test it, and how long?
ср, 15 мая 2019 г., 23:50 Philipp Schönberger [email protected]:
hi, i tested the v4.5.5 on Ubuntu Linux and used java 8.211:
$java -version java version "1.8.0_211" Java(TM) SE Runtime Environment (build 1.8.0_211-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.211-b12, mixed mode)
memory seems fine but cpu usage still goes over the roof and slows the pc down after some time (1 core with ~95% use no mater what).
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cisco-system-traffic-generator/trex-stateless-gui/issues/101?email_source=notifications&email_token=AINDXDI57TTXJWP7LKX3KITPVQ5MZA5CNFSM4GFDRAR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVPIQCQ#issuecomment-492734474, or mute the thread https://github.com/notifications/unsubscribe-auth/AINDXDMO2ESB44IC464NQQDPVQ5MZANCNFSM4GFDRARQ .
i created a small script to plot it over time and gather some infos i started trex and created two simple ip/udp traffic with minimal pps and started it and let this running for some time.
trafic.yml
---
- name: "ip/udp-1"
stream:
action_count: 0
enabled: true
flags: 2
flow_stats:
enabled: true
rule_type: "latency"
stream_id: 2
isg: 0.0
mode:
rate:
type: "pps"
value: 1.0
type: "continuous"
total_pkts: 2
pkts_per_burst: 0
ibg: 0.0
count: 1
packet:
binary: "AAAAAAAAAAAAAAAAgQAAZAgARQAATgTSAAB/Ece3CgEtCgoBLgoAZABkADIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
meta: "eyJwcm90b2NvbF9zZWxlY3Rpb24iOnsiaXNfdGFnZ2VkX3ZsYW5fc2VsZWN0ZWQiOnRydWUsImlzX2lwdjRfc2VsZWN0ZWQiOnRydWUsImlzX3RjcF9zZWxlY3RlZCI6ZmFsc2UsImlzX3VkcF9zZWxlY3RlZCI6dHJ1ZSwiaXNfcGF0dGVybl9zZWxlY3RlZCI6ZmFsc2UsImZyYW1lX2xlbmd0aF90eXBlIjoiRml4ZWQiLCJmcmFtZV9sZW5ndGgiOiIxMDAiLCJtYXhfbGVuZ3RoIjoiMTUxOCIsIm1pbl9sZW5ndGgiOiI2NCJ9LCJldGhlcm5ldCI6eyJ0eXBlIjoiMDgwMCIsImlzX292ZXJyaWRlIjpmYWxzZX0sIm1hYyI6eyJzb3VyY2UiOnsiYWRkcmVzcyI6IjAwOjAwOjAwOjAwOjAwOjAwIiwiY291bnQiOiIxNiIsIm1vZGUiOiJUUmV4IENvbmZpZyIsInN0ZXAiOiIxIn0sImRlc3RpbmF0aW9uIjp7ImFkZHJlc3MiOiIwMDowMDowMDowMDowMDowMCIsImNvdW50IjoiMTYiLCJtb2RlIjoiRml4ZWQiLCJzdGVwIjoiMSJ9fSwiaXB2NCI6eyJzb3VyY2UiOnsiYWRkcmVzcyI6IjEwLjEuNDUuMTAiLCJjb3VudCI6IjE2IiwibW9kZSI6IkZpeGVkIiwic3RlcCI6IjEifSwiZGVzdGluYXRpb24iOnsiYWRkcmVzcyI6IjEwLjEuNDYuMTAiLCJjb3VudCI6IjE2IiwibW9kZSI6IkZpeGVkIiwic3RlcCI6IjEifX0sInRjcCI6eyJpc19vdmVycmlkZV9kc3RfcG9ydCI6ZmFsc2UsImNoZWNrc3VtIjoiQjNFMyIsImRzdF9wb3J0IjoiMTAyNSIsInNyY19wb3J0IjoiMTAyNSIsImlzX292ZXJyaWRlX3NyY19wb3J0IjpmYWxzZSwiYWNrX251bWJlciI6IjAiLCJpc19vdmVycmlkZV9jaGVja3N1bSI6ZmFsc2UsInVyZ2VudF9wb2ludGVyIjoiMCIsImlzX3VyZyI6ZmFsc2UsImlzX2FjayI6ZmFsc2UsImlzX3BzaCI6ZmFsc2UsImlzX3JzdCI6ZmFsc2UsImlzX3N5bmMiOmZhbHNlLCJpc19maW4iOmZhbHNlLCJzZXF1ZW5jZV9udW1iZXIiOiIxMjkwMTgiLCJ3aW5kb3ciOiIxMDI0In0sInVkcCI6eyJpc19vdmVycmlkZV9kc3RfcG9ydCI6dHJ1ZSwiY2hlY2tzdW0iOiJGRkJBIiwiZHN0X3BvcnQiOiIxMDAiLCJzcmNfcG9ydCI6IjEwMCIsImlzX292ZXJyaWRlX2xlbmd0aCI6ZmFsc2UsImlzX292ZXJyaWRlX3NyY19wb3J0Ijp0cnVlLCJpc19vdmVycmlkZV9jaGVja3N1bSI6ZmFsc2UsImxlbmd0aCI6IjI2In0sInBheWxvYWQiOnsidHlwZSI6IkZpeGVkIFdvcmQiLCJwYXR0ZXJuIjoiMDAifSwidmxhbiI6eyJjZmkiOiIwIiwiaXNfb3ZlcnJpZGVfdHBfaWQiOmZhbHNlLCJ0cF9pZCI6IkZGRkYiLCJ2X2lkIjoiMTAwIiwicHJpb3JpdHkiOiIwIn0sImFkdmFuc2VkX3Byb3BlcnRpZXMiOnsiY2FjaGVfdmFsdWUiOiI1MDAwIiwiY2FjaGVfc2l6ZV90eXBlIjoiRW5hYmxlIn0sIm1vZGVsX3ZlcnNpb24iOiIxLjAifQ=="
model: ""
self_start: true
advanced_mode: false
vm:
split_by_var: ""
instructions: []
cache_size: 5000
stream_id: 0
- name: "ip/udp-2"
stream:
action_count: 0
enabled: true
flags: 2
flow_stats:
enabled: true
rule_type: "latency"
stream_id: 3
isg: 0.0
mode:
rate:
type: "pps"
value: 1.0
type: "continuous"
total_pkts: 2
pkts_per_burst: 0
ibg: 0.0
count: 1
packet:
binary: "AAAAAAAAAAAAAAAAgQAAZAgARQAATgTSAAB/Ece3CgEtCgoBLgoAyADIADIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
meta: "eyJwcm90b2NvbF9zZWxlY3Rpb24iOnsiaXNfdGFnZ2VkX3ZsYW5fc2VsZWN0ZWQiOnRydWUsImlzX2lwdjRfc2VsZWN0ZWQiOnRydWUsImlzX3RjcF9zZWxlY3RlZCI6ZmFsc2UsImlzX3VkcF9zZWxlY3RlZCI6dHJ1ZSwiaXNfcGF0dGVybl9zZWxlY3RlZCI6ZmFsc2UsImZyYW1lX2xlbmd0aF90eXBlIjoiRml4ZWQiLCJmcmFtZV9sZW5ndGgiOiIxMDAiLCJtYXhfbGVuZ3RoIjoiMTUxOCIsIm1pbl9sZW5ndGgiOiI2NCJ9LCJldGhlcm5ldCI6eyJ0eXBlIjoiMDgwMCIsImlzX292ZXJyaWRlIjpmYWxzZX0sIm1hYyI6eyJzb3VyY2UiOnsiYWRkcmVzcyI6IjAwOjAwOjAwOjAwOjAwOjAwIiwiY291bnQiOiIxNiIsIm1vZGUiOiJUUmV4IENvbmZpZyIsInN0ZXAiOiIxIn0sImRlc3RpbmF0aW9uIjp7ImFkZHJlc3MiOiIwMDowMDowMDowMDowMDowMCIsImNvdW50IjoiMTYiLCJtb2RlIjoiRml4ZWQiLCJzdGVwIjoiMSJ9fSwiaXB2NCI6eyJzb3VyY2UiOnsiYWRkcmVzcyI6IjEwLjEuNDUuMTAiLCJjb3VudCI6IjE2IiwibW9kZSI6IkZpeGVkIiwic3RlcCI6IjEifSwiZGVzdGluYXRpb24iOnsiYWRkcmVzcyI6IjEwLjEuNDYuMTAiLCJjb3VudCI6IjE2IiwibW9kZSI6IkZpeGVkIiwic3RlcCI6IjEifX0sInRjcCI6eyJpc19vdmVycmlkZV9kc3RfcG9ydCI6ZmFsc2UsImNoZWNrc3VtIjoiQjNFMyIsImRzdF9wb3J0IjoiMTAyNSIsInNyY19wb3J0IjoiMTAyNSIsImlzX292ZXJyaWRlX3NyY19wb3J0IjpmYWxzZSwiYWNrX251bWJlciI6IjAiLCJpc19vdmVycmlkZV9jaGVja3N1bSI6ZmFsc2UsInVyZ2VudF9wb2ludGVyIjoiMCIsImlzX3VyZyI6ZmFsc2UsImlzX2FjayI6ZmFsc2UsImlzX3BzaCI6ZmFsc2UsImlzX3JzdCI6ZmFsc2UsImlzX3N5bmMiOmZhbHNlLCJpc19maW4iOmZhbHNlLCJzZXF1ZW5jZV9udW1iZXIiOiIxMjkwMTgiLCJ3aW5kb3ciOiIxMDI0In0sInVkcCI6eyJpc19vdmVycmlkZV9kc3RfcG9ydCI6dHJ1ZSwiY2hlY2tzdW0iOiJGRkJBIiwiZHN0X3BvcnQiOiIyMDAiLCJzcmNfcG9ydCI6IjIwMCIsImlzX292ZXJyaWRlX2xlbmd0aCI6ZmFsc2UsImlzX292ZXJyaWRlX3NyY19wb3J0Ijp0cnVlLCJpc19vdmVycmlkZV9jaGVja3N1bSI6ZmFsc2UsImxlbmd0aCI6IjI2In0sInBheWxvYWQiOnsidHlwZSI6IkZpeGVkIFdvcmQiLCJwYXR0ZXJuIjoiMDAifSwidmxhbiI6eyJjZmkiOiIwIiwiaXNfb3ZlcnJpZGVfdHBfaWQiOmZhbHNlLCJ0cF9pZCI6IkZGRkYiLCJ2X2lkIjoiMTAwIiwicHJpb3JpdHkiOiIwIn0sImFkdmFuc2VkX3Byb3BlcnRpZXMiOnsiY2FjaGVfdmFsdWUiOiI1MDAwIiwiY2FjaGVfc2l6ZV90eXBlIjoiRW5hYmxlIn0sIm1vZGVsX3ZlcnNpb24iOiIxLjAifQ=="
model: ""
self_start: true
advanced_mode: false
vm:
split_by_var: ""
instructions: []
cache_size: 5000
stream_id: 2
start.sh output
###############
CPU:
model name : Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
###############
Mem:
total used free shared buffers cached
Mem: 30918 11050 19868 289 106 2243
-/+ buffers/cache: 8699 22218
Swap: 32518 658 31860
###############
java:
java version "1.8.0_211"
Java(TM) SE Runtime Environment (build 1.8.0_211-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.211-b12, mixed mode)
###############
trex version
version=v4.5.5-0-gb80ac59
###############
start trex:
###############
pid is 18080
start.sh to get cpu and mem
#!/bin/bash
echo "###############"
echo "CPU:"
cat /proc/cpuinfo | grep "model name"| uniq
echo ""
echo "###############"
echo "Mem:"
free -m
echo ""
echo "###############"
echo "java:"
java -version
echo ""
echo "###############"
echo "trex version"
unzip -p trex-stateless-gui.jar version.properties
echo ""
echo ""
echo "###############"
echo "start trex:"
java -jar trex-stateless-gui.jar > trex.log &
pid=$!
trap "echo '' kill $pid ! ; kill $pid ; exit -1; ./plot.sh" SIGINT SIGTERM
START_TIME=$SECONDS
echo ""
echo "###############"
echo "pid is $pid"
rm trex_stats.dat
touch trex_stats.dat
while true
do
ELAPSED_TIME=$(($SECONDS - $START_TIME))
echo -n "$ELAPSED_TIME ">>trex_stats.dat
ps -o pid,pcpu,rss,pmem -p $pid | tail -n1 >> trex_stats.dat
sleep 10
if [[ $? -ne 0 ]]
then
break
fi
done
plot.sh to get graph
#!/bin/bash
gnuplot << eor
set title 'trex cpu/mem usage'
set ylabel 'cpu %'
set y2label 'mem kilobytes'
set xlabel 'time in sec'
set grid
set terminal pngcairo size 2000,1000
set output 'cpu.png'
set datafile separator whitespace
set style line 1 \
linecolor rgb '#0060ad' \
linetype 1 linewidth 2 \
pointtype 7 pointsize 0.5
set style line 2 \
linecolor rgb '#ff60ad' \
linetype 2 linewidth 2 \
pointtype 7 pointsize 0.5
set ytics 10 nomirror
set y2tics
set key outside below
plot 'trex_stats.dat' using 1:3 title "CPU" with linespoints linestyle 1 , \
'trex_stats.dat' using 1:4 title "Mem" with linespoints linestyle 2 axes x1y2
eor