`feed-me/feeds/queue` should process feeds in sequence

[timkelty]

Description

• https://github.com/craftcms/feed-me/issues/1584
• https://craftcms.slack.com/archives/C040W72JA5N/p1757536676035999

[linear[bot]]

PT-2947

[timkelty]

@bryantAXS @pixelmachine could you test this and ensure it addresses your issues?

# for Craft 4.x
composer require craftcms/feed-me 5.x-dev

# for Craft 5.x
composer require craftcms/feed-me 6.x-dev

[angrybrad]

Fixed in https://github.com/craftcms/feed-me/issues/1695 and will be included in the next release!

[pixelmachine]

@timkelty tested in Craft 4 and I don't think this is working as we needed.

Say I have 6 feeds of different lengths that need to run sequentially and I trigger them all in sequence, my queue will initially look like this:

• Feed A
• Feed B
• Feed C (Batch 1 of 5)
• Feed D (Batch 1 of 3)
• Feed E (Batch 1 of 4)
• Feed F

The batched run order will then be:

• A->B->C1->D1->E1->F
• C2->D2->E2
• C3->D3->E3
• C4->E4
• C5

Essentially they're still running out of order and anything after C that references previous feeds may fail.

[i-just]

@pixelmachine, can you please let me know how exactly you trigger your feeds? Is it via CLI? If so, what is the full command you use?

[pixelmachine]

@i-just I'm actually just running them with a web request that loops over the feeds to trigger each. In my case it's run manually by the user as needed.

[cmalven]

Is it possible to queue feeds sequentially directly through a module? In my specific case, I'd like to create a dashboard widget that allows users to easily queue up certain feeds that need to be run in a specific order.

[timkelty]

@cmalven right now the sequential logic is bound to the CLI and associate queue job, but you could trigger that like:

Craft::$app->runAction('feed-me/feeds/queue', [1, 2, 3]);

[cmalven]

@cmalven right now the sequential logic is bound to the CLI and associate queue job, but you could trigger that like:

Craft::$app->runAction('feed-me/feeds/queue', [1, 2, 3]);

@timkelty Thank you for the tip! I had tried that but I'm getting a Page not found. -> Caused by: Invalid Route - Unable to resolve the request: feed-me/feeds/queue in /var/www/html/vendor/yiisoft/yii2/base/Controller.php at line 149 when I try to run that in my controller. Should there be anything special I need to do to get this to work?

[timkelty]

If you're running it from a controller, try just doing $this->run('feed-me/feeds/queue', [1, 2, 3])

[cmalven]

@timkelty Still getting a Page not found, this time Unable to resolve the request "import/feed-me/feeds/queue". where import is the namespace of the module that is calling $this->run('feed-me/feeds/queue', [1, 3, 2]);

[timkelty]

@cmalven my bad, you need a leading slash:

$this->run('/feed-me/feeds/queue', [1, 2, 3])

[cmalven]

@timkelty really appreciate you sticking with me on this!

$this->run('/feed-me/feeds/queue', [1, 2, 3]) still leads to an Invalid Route, but interestingly this time Unable to resolve the request: feed-me/feeds/queue - no quotes around feed-me/feeds/queue

[timkelty]

@cmalven try an array with a single, comma-separated value:

class MyController extends yii\console\Controller
{
    public function actionIndex()
    {
        return $this->run('/feed-me/feeds/queue', ['1,2,3']);
    }
}

Seems to be working over here. Let me know.

[cmalven]

Thanks @timkelty. I'm still getting the same error with your code, but I noticed that your example uses yii\console\Controller but mine is using craft\web\Controller because I'm calling it from a widget. Would that make a difference?

[timkelty]

Ah, yep. $this->run assumes calling from a console controller.

because I'm calling it from a widget

In that case, you can just all the method directly:

(new \craft\feedme\console\controllers\FeedsController('feeds', Craft::$app))->actionQueue('1,2');

[cmalven]

@timkelty This worked! Thank you for enduring all of that back and forth! Really appreciate it.