pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[Bug] Java Client : Pulsar Java client should be able to configure compression for message rejection in the DLQ

Open lgringo opened this issue 1 year ago • 2 comments

Search before asking

  • [X] I searched in the issues and found nothing similar.

Version

  • Pulsar 3.0.1
  • Java Client 3.0.1

Minimal reproduce step

. Configure a producer with LZ4 compression enabled . Configure a consumer with DLQ. . Reject a very big message (negative ack)

What did you expect to see?

Message sent to the DLQ

What did you see instead?

Execption : o.a.pulsar.client.impl.ConsumerImpl : [persistent://zzz/mailbox/message-event] Failed to send DLQ message to zzz/mailbox/message-event-recorder-DLQ for message id 64563:0:-1 java.util.concurrent.CompletionException: org.apache.pulsar.client.api.PulsarClientException$InvalidMessageException: Message size is bigger than 20971520 bytes

NB: Max Message size and netty are set 20Mb.

Anything else?

Maybe related to :

  • https://github.com/apache/pulsar/issues/19463
  • https://github.com/apache/pulsar/pull/20959

Are you willing to submit a PR?

  • [ ] I'm willing to submit a PR!

lgringo avatar Jan 23 '24 08:01 lgringo

Any ideas @poorbarcode @315157973 @liudezhi2098 ?

lhotari avatar Jan 23 '24 10:01 lhotari

IMO, Maybe we can force disabled batch to ensure the original status of failed messages. PIP-293 looks too complicated.

315157973 avatar Jan 24 '24 12:01 315157973