Henare Degan
Henare Degan
I just needed to test this and this is the rough process I went though: - Start scraper - Wait for a container to boot - [make job crash] I...
Very similar issue: #845.
This has recently been starting to happen hundreds and sometimes thousands of times an hour so I'm pretty sure it's related to a recent change. It just affects the backend...
Would you believe I just freed 8 GB by deleting 632,284 of these files?!
Something is going on with the queue now. We have a backlog of about 200 scrapers. There doesn't seem to be many really long running scrapers but I did add...
> Now there's 17 running containers but only 2 sidekiq jobs. Running rake app:emergency:show_queue_run_inconsistencies doesn't show any inconsistencies. Huh? It's because they're on the retry queue. OK, I think I...
There are 13 scrapers that have been running for over 5 hours. A couple of them seem to routinely run for a day so I've disabled auto-run and added them...
>That happens and the jobs end up way back in the retry queue. Sometimes they get backed off to having hours before a next retry. I'm not sure why they...
@thdoan Hi Tom, welcome :wave: Thanks for this really good issue write-up :star: I think this is something we'd welcome in a PR. Related: #1043