boa icon indicating copy to clipboard operation
boa copied to clipboard

Script var/xdrago/daily.sh runs for an extremely long time and multiple executions

Open namontague opened this issue 4 years ago • 7 comments

Hi,

The var/xdrago/daily.sh runs for a very long time (days). In my case this runs for longer than a single day and is then triggered again the following day while the script it already running as it has not finished. This is causing a lot of CPU processing power to be used up until there is no resources left to run the web server.

We have just migrated from an older version of BOA 2.7 to 4.0.1. Our setup consists of over 2,000 websites and based on a smaller setup(110 websites) we have, I've estimated the daily.sh script would take 181 hours! to complete for these 2,000 websites.

Has anyone overcome this issue or know of any ideas to solve this? Also is there any documentation about these crontab scripts that BOA runs?

Server Info: Debian 9 BOA 4.0.1 32GB Ram 8 CPUs PHP 7.3

namontague avatar Jun 23 '20 07:06 namontague

Please upgrade to head, the stable 4.0.1 is over one year old and shouldn't be used to avoid issues.

We have reduced (in head) the number of tasks this script runs, and it will be split into several smaller scripts, but it's indeed still not designed to be used on systems with very large number of sites hosted. You could disable it by adding file with this contents to /etc/etc/cron.hourly/stop_daily

#!/bin/bash

touch /var/run/daily-fix.pid

Then run chmod 700 /etc/etc/cron.hourly/stop_daily

omega8cc avatar Jul 22 '20 12:07 omega8cc

Thanks for the update.

Will look at upgrading.

However, In the short term what is the downside to disabling this script? I cant find any documentation to find out exactly the purpose of this script.

With this disabled what should I be doing manually that this script did that is significantly important to the running of BOA?

namontague avatar Jul 22 '20 13:07 namontague

This scripts does a lot of useful things, it clears temporary files/leftovers which may cause problems for tasks like clone/migrate, it fixes permissions on all sites (it's actually no longer required feature with recent BOA head, which fixes the permissions on standard Verify for site and platform). It also checks and updates Let's Encrypt certificates if enabled in any site.

omega8cc avatar Jul 22 '20 13:07 omega8cc

Latest version no longer disables/enables any modules, though, which was the most time consuming task.

omega8cc avatar Jul 22 '20 13:07 omega8cc

In searching for upgrade information you mention that HEAD should never be used in a production environment. Is this no longer the case?

If so can you please direct me to information on upgrading from stable to head as I can not find this in the UPGRADE.txt file.

namontague avatar Jul 22 '20 13:07 namontague

Yes, it's no longer valid warning, but ironically we need to release one more stable before we can remove it and switch to rolling release policy.

To upgrade to head the procedure is the same, just use up-head instead of up-stable, as explained in https://github.com/omega8cc/boa/blob/master/docs/UPGRADE.txt#L127

We are running head in production for years.

omega8cc avatar Jul 22 '20 13:07 omega8cc

Thank you for your help.

namontague avatar Jul 22 '20 14:07 namontague

Closing old tickets related to issues fixed already. Feel free to reopen if something is still relevant. Thank you!

omega8cc avatar May 08 '23 19:05 omega8cc