`gpbackup` sendmail step always fails when running in `crontab`

[CC-Hsu]

Hi, Team,

When executing gpbackup / gprestore command directly with gp_email_contacts.yaml setup, we can receive email after command finish. https://gpdb.docs.pivotal.io/backup-restore/1-20/admin_guide/managing/backup-gpbackup.html#topic_qwd_d5d_tbb

However, when putting the same command in crontab (running with gpadmin account), mail sending always fails, with the following error.

20210716:02:08:29 gpbackup:gpadmin:gpdb:157501-[INFO]:-/home/gpadmin/gp_email_contacts.yaml list found, /data/master/gpseg-1/backups/20210716/20210716020827/gpbackup_20210716020827_report will be sent
20210716:02:08:29 gpbackup:gpadmin:gpdb:157501-[WARNING]:-Unable to send email report: bash: line 68: sendmail: command not found

Can you fix the issue from gpbackup source code?

Regards.

[jimmyyih]

From the documentation:

Note: The UNIX mail utility must be running on the Greenplum Database host and must be configured to allow the Greenplum superuser (gpadmin) to send email. Also ensure that the mail program executable is locatable via the gpadmin user's $PATH.

Try verifying that the sendmail utility is properly set in the PATH environment variable for your crontab entry or make sure that it's installed with yum install sendmail # for CentOS/RedHat.

[CC-Hsu]

Hi, jimmyyih,

We've checked our customer's environment already install the package, and manually execute gpbackup command can trigger the mail report.

But it just doesn't work in cronjob.

It seems that the executable /usr/sbin/crontab by default is not searched inside crontab of gpadmin account.

So maybe your team should consider to write full sendmail executable path in gpbackup/gprestore.

[jimmyyih]

It seems that the executable /usr/sbin/crontab by default is not searched inside crontab of gpadmin account.

Yes, crontab starts a very minimal shell and does not source the user profile. The admin must configure the environment properly for the crontab entry.

Examples:

# source greenplum_path.sh and add /usr/sbin (location of sendmail utility) to the path
* * * * * source /usr/local/gpdb/greenplum_path.sh && PATH=/usr/sbin:$PATH gpbackup --dbname postgres

# run command in a shell that has sourced the user profile
* * * * * bash -lc "gpbackup --dbname postgres"

# or run a shell script that sources greenplum_path.sh and exports the $PATH to include sendmail utility before running gpbackup
* * * * * bash myscript.sh

The above solutions would be the easiest way to fix your issue without any patches. However, we could consider adding a path_to_sendmail configuration parameter in the email configuration yaml to help in unlikely scenarios where the $PATH environment variable cannot be set.

If possible, it would be helpful to provide the crontab entry and the output of which sendmail so that we can reproduce the issue and help further.

[CC-Hsu]

Hi, jimmyyih,

Our crontab entry is exactly what you show in the third case, just calling a shell script which wraps gpbackup call.

For the location of sendmail in CentOS7.x (also in RHEL7.x but it's on our customer's environment),

[gpadmin@gpdb ~]$ which sendmail
/usr/sbin/sendmail
[gpadmin@gpdb ~]$ 

Your idea path_to_sendmail is also a viable option for customers; it would be great to implement it.

[jimmyyih]

If it's running a shell script, simply adding export PATH=$PATH:/usr/sbin before the gpbackup call should be a viable workaround then for this particular customer of yours.

As for the path_to_sendmail configuration option, we'll see what we can do in a future release.

[CC-Hsu]

Hi, jimmyyih,

Got it.

Thanks.