facerecognition icon indicating copy to clipboard operation
facerecognition copied to clipboard

SQLSTATE[HY000] MySQL server has gone away on "CreateClustersTask"

Open isaacolsen94 opened this issue 3 years ago • 7 comments

Not sure how to proceed in troubleshooting this issue. It happened twice for me and I verified my database is running. Everything else that relies on the database seems fine except this. Would you be able to help me diagnose this?

Expected behaviour

Background task should complete without errors

Actual behaviour

The task has failed multiple times at the 6/10 "CreateClustersTask"

image

Server configuration

  • Operating system: Ubuntu

isaacolsen94 avatar Dec 15 '21 01:12 isaacolsen94

Hi @isaacolsen94 It seems, you have so many faces, and some queries are slow. 😞 Try: https://stackoverflow.com/questions/7942154/mysql-error-2006-mysql-server-has-gone-away

matiasdelellis avatar Dec 15 '21 11:12 matiasdelellis

That did the trick! I also doubled the values of a few of the timeouts/cache settings and it was able to complete properly. Thank you so much for your help!!!

isaacolsen94 avatar Dec 16 '21 01:12 isaacolsen94

@isaacolsen94 Do you remember what settings you have changed?

JojoDevel avatar Mar 20 '22 14:03 JojoDevel

Adding this to my mariadb config solved the issue

innodb_log_file_size=512M

JojoDevel avatar Mar 20 '22 14:03 JojoDevel

I didn't remember which options I changed, but I will give this one a try! Thanks for sharing your fix!

I think I went through the recommendations in the mentioned article and doubled each resource call out. Probably wasn't the best move but it seemed to work. But I haven't verified recently.

isaacolsen94 avatar Mar 20 '22 15:03 isaacolsen94

This can be solved by adjusting the following variable in mariadb config:

wait_timeout=86400

It means MySQL will wait for 24h your process to be done before giving up. Increase at will (up to 31536000). Just to illustrate, I'm testing my server right now, and it's been taking 8h to process 150K faces. So far! :laughing:

ftrentini avatar Apr 08 '22 21:04 ftrentini