SQS batch method in SQSQueue::bulk

[AbdullahZN]

Hi,

Since SQS has a sendMessageBatch action. Would it be ok to implement a bulk method for SQSQueue with the following:

class SqsQueue
{    
    public function bulk($jobs, $data = '', $queue = null)
    {
        $entries = [];
        foreach ($jobs as $id => $job) {
            $entries[] = [
                'Id' => $id,
                'MessageBody' => $this->createPayload($job, $data),
            ];
        }

        return $this->sqs->sendMessageBatch(
            [
                'QueueUrl' => $this->getQueue($queue),
                'Entries' => $entries,
            ]
        );
    }
}

I'd be glad to submit a PR

[alexbowers]

What would be the API change for somebody to bulk queue jobs?

Currently you dispatch jobs based on either dispatch(new JobName) or JobName::dispatch.

What is the proposed change to send many at once?

[sisve]

@alexbowers There are many more ways to dispatch jobs, the most relevant one would be Queue::bulk($arrayOfJobs, $data, $queue);

[AbdullahZN]

Yes exactly, you'll need to explicitly tell that you want to batchPush.

Using bulk will work for other type of queues as well since the bulk method is implemented in Illuminate\Queue\Queue eg: you are using SyncQueue locally & SqsQueue on production

[alexbowers]

Well damn, @sisve That's going to be crazy helpful to me. Didn't realise you could do bulk already. Is this documented? :o

[sisve]

@alexbowers Kind of. Does the api docs count? The contract got the bulk method in Laravel 5.4, but the implementation has supported it since Laravel 4.0.7. The base Queue class implements it by looping through the jobs and calling push() on them one-by-one, so all queues "support" the method, they are perhaps not just optimized for it.

https://laravel.com/api/5.8/Illuminate/Queue/Queue.html#method_bulk

[atymic]

Bumping this issue. I plan to override the existing method in my app so it does the following:

• 0 messages -> ignore
• 1 - 10 messages under 256kb -> batch dispatch
• 10+ messages -> loop, batch dispatching when size or count threshold is met.

Would laravel accept a PR?

[atymic]

For anyone wondering, i implemented this, async bulk dispatch :) https://packagist.org/packages/atymic/laravel-bulk-sqs-queue