laravel-queue-rabbitmq
laravel-queue-rabbitmq copied to clipboard
Option to GZIP messages on queue
@adm-bome @M-Porter @mattgrande
Would like your opinion on adding this as an option.
When doing more intensive usage of RabbitMQ, memory management on the broker can become an issue. At work, we've had good success extending the classes in this library to gzip the message bodies on the queues. This can in some cases have a very dramatic and beneficial effect on the cluster.
I think more apps could benefit from this, and this library could (should?) offer an easy option to turn this on.
A previous PR I had made for this (https://github.com/vyuldashev/laravel-queue-rabbitmq/pull/447) enabled it as a connection level setting.
However at work, we've done this on a per-job basis instead.
That PR also only allowed a single compression algorithm (gzip) whereas if I made this to merge in this lib now, I would prefer we use a configurable compression algorithm.
Before committing more work to this though, I'd like to gather your thoughts on whether yes or no this should be part of this library.
Sounds fine by me. I would like to see some proposal in a branch.
As I understand this now: Zip the payload of a AMQPMessage, when creating and Unzip when retrieved, right?
Some shout-outs
'connections' => [
// ...
'rabbitmq' => [
// ...
'options' => [
'compressor' => 'gzip' // or a Compressor::class
],
],
// ...
],
When compressor
is omitted, the default should be 'false' in QueueConfig
and should be to not compress or decompress. (disabled by default)
When compressor
is set anything other then false
, for example to gzip
, the correct compression should be applied.
Optional: (like to see this) Maybe create a abstract CompressorClass with compress/decompress methods/interface, and a compressorFactory, so alternate compressors can be added by config.
Implement a:
- NullCompressor::class, as a default in the
QueueConfig
(won't compress) - GzipCompressor:class, (chosen in the factory when 'gzip' is set into the connections::rabbitmq::options:compressor config.)
- ...
- ...