Queue jobs not running under certain circumstances

[jmauzyk]

I noticed some odd behavior in which it seemed like Async Queue wasn't successfully running queue jobs when pushed. At first, it seemed like there wasn't much rhyme or reason to why this was happening, but after some digging I may have determined the cause.

I created a bare-bones queue job and console command for testing:

<?php

namespace modules\sitemodule\jobs;

use Craft;
use craft\queue\BaseJob;

/**
 * Test queue job
 */
class Test extends BaseJob
{
    function execute($queue): void
    {
        Craft::info('Successfully ran Test queue job');
        $this->setProgress($queue, 1);
    }

    protected function defaultDescription(): ?string
    {
        return 'Running test';
    }
}

<?php

namespace modules\sitemodule\console\controllers;

use Craft;
use craft\console\Controller;
use modules\sitemodule\jobs\Test;
use yii\console\ExitCode;

/**
 * Test controller
 */
class TestController extends Controller
{
    public $defaultAction = 'index';

    public function options($actionID): array
    {
        $options = parent::options($actionID);
        switch ($actionID) {
            case 'index':
                // $options[] = '...';
                break;
        }
        return $options;
    }

    /**
     * site-module/test command
     */
    public function actionIndex(): int
    {
        Craft::$app->getQueue()->push(new Test());
        return ExitCode::OK;
    }
}

When running craft site-module/test as defined above, the Test queue job would be queued and in Pending status, but would never be reserved or executed.

However, if I added a sleep statement to the console command

        Craft::$app->getQueue()->push(new Test());
        sleep(5);
        return ExitCode::OK;

the Test queue job would successfully reserve and execute asynchronously, the expected behavior for Async Queue.

This mirrors the behavior I was experiencing with the original console command that prompted me to investigate this: executions that had several elements to process were longer-running, providing enough time for Aysnc Queue to successfully run queue jobs pushed during that time, but executions with a single element were terminated almost immediately after the queue job was pushed, resulting in the job not being executed.

I'm not intimately familiar with the inner workings of this plugin, however, it seems like:

1. The part of this plugin that makes queue jobs run automatically uses some asynchronous logic, and
2. It's possible for that logic to be interrupted if the PHP CLI process is terminated before it completes, preventing the pushed job from running automatically

Are you able to confirm whether this is the case and, if so, find a resolution for this issue?

In this instance, the behavior is occurring within a console request, but I'm wondering if similar issues with queue jobs pushed during site requests might be caused by the PHP-FPM process terminating before the plugin's asynchronous logic completes: https://github.com/ostark/craft-async-queue/issues/64

I appreciate you taking the time to look into this, and apologize if I'm off-base on any of my assumptions here.

[ostark]

thanks @jmauzyk for the detailed report :-) hopefully I fill find some time soon to have a deeper look.