[Question] Gh-ost hits fatal load even if max-load is configured

[edeak]

I was running the following gh-ost command:

./gh-ost \
--max-load=Threads_running=50 \
--critical-load=Threads_running=5000 \
--chunk-size=10000 \
--throttle-control-replicas="mysql-replica-1" \
--max-lag-millis=10000 \
--user="******" \
--password="******" \
--host=localhost \
--database="******" \
--table="*******" \
--verbose \
--alter="*******" \
--switch-to-rbr \
--cut-over=default \
--exact-rowcount \
--concurrent-rowcount \
--default-retries=120 \
--panic-flag-file=/tmp/ghost.panic.flag \
--postpone-cut-over-flag-file=/tmp/ghost.postpone.flag \
--throttle-flag-file=/tmp/throttle \
--execute | tee /tmp/ghost.txt

and around 90% progress I've got a fatal error of hitting the critical load. Maybe I'm overlooking something, but I'd expect gh-ost to apply throttling when number of threads hit the number configured with --max-load. Also, which threads are used for max-load and critical-load calculation?

[tomkrouper]

Do you have monitoring on Threads_running on the hosts in question? The only thing that I can think of off the top of my head is a sudden jump of threads_running. You could add --critical-load-interval-millis to have it try a second time to see if that helps.