Health check for send queue

[boonebgorges]

It may occasionally happen that the send queue is interrupted for some reason - especially the 'immediate' queue. When this happens, there's currently no mechanism that restarts the process, or checks that it finished. I'm thinking there could be a regular health check that would do something like the following:

1. Every so often (15 minutes? 1 hour?), check for 'immediate' items in the queue that are older than, say, 10 minutes. Anything newer may be in the process of being sent.
2. For each unique activity_id, trigger a send batch

[boonebgorges]

I had to whip up something quick for a client having the problem. It looks like this:

add_filter( 'cron_schedules', function ( $schedules ) {
	$schedules['five_minutes'] = array(
		'interval' => 5 * MINUTE_IN_SECONDS,
		'display'  => esc_html__( 'Every Five Minutes' ),
	);

	return $schedules;
 } );

function cac_bpges_health_check() {
	global $wpdb;

	$table_name = bp_core_get_table_prefix() . 'bpges_queued_items';

	$before = date( 'Y-m-d H:i:s', time() - ( 5 * MINUTE_IN_SECONDS ) );
	$after  = date( 'Y-m-d H:i:s', time() - ( DAY_IN_SECONDS ) );

	$sql = $wpdb->prepare( "SELECT DISTINCT(activity_id), date_recorded FROM $table_name WHERE date_recorded < %s AND date_recorded > %s AND type = 'immediate' LIMIT 5", $before, $after );

	$results = $wpdb->get_results( $sql );

	foreach ( $results as $result ) {
		// _b( 'Sending ping for ' . $result->activity_id );

		// Trigger the batch process.
		bpges_send_queue()->data( array(
			'type'        => 'immediate',
			'activity_id' => $result->activity_id,
		) )->dispatch();
	}
}
add_action( 'bpges_health_check', 'cac_bpges_health_check' );

Something like this is probably adaptable and appropriate for BPGES itself, but a few things should be filterable or configurable: the number of items per batch (hardcoded as 5), the cron schedule (hardcoded as every 5 minutes here), the older-than and newer-than dates (hardcoded to 5 minutes and 24 hours)

[boonebgorges]

Related, it would be helpful to have a health-check for digests, but the logic there would be a bit more involved. You would only want to check for unsent items that are older than the most recent digest-send (but newer than, say, 3x the digest period - three days or three weeks?). So this might mean a mechanism other than an every-five-minute cron job, like perhaps the digest queue process schedules its own one-time health check at the same time that it begins running. This needs further thought.

[boonebgorges]

Probably related: https://wordpress.org/support/topic/bpges_queued_items-ballooning-admin-ajax-timout/#post-13382790