pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

The consumer clients created by the Proxy using the WebSocket protocol may cause request blocking.

Open zifengmo opened this issue 2 years ago • 1 comments

Search before asking

  • [X] I searched in the issues and found nothing similar.

Motivation

In our usage scenario, the ability to create subscriptions directly through the consumer client has been disabled. This leads to a problem where the Proxy creates consumer clients for consuming topics via WebSocket. In certain exceptional cases, these clients will keep retrying to connect to the broker, such as when encountering a SubscriptionNotFoundException ("Subscription does not exist"). However, this exception is considered retryable, causing the Proxy to continuously attempt to connect to the broker for subscription. Because this exception leads to retries and bypasses the consumer quantity verification, it causes the Proxy to continuously retry the subscription, blocking normal requests. Therefore, it is desired to have special handling for consumer clients created through the Proxy, or to subtract something specific for retryable exceptions, in order to avoid situations like the one mentioned above where such exceptions trigger retries in the Proxy.

Code Example: public static boolean isRetriableError(Throwable t) { if (t instanceof AuthorizationException || t instanceof InvalidServiceURL || t instanceof InvalidConfigurationException || t instanceof NotFoundException || t instanceof IncompatibleSchemaException || t instanceof TopicDoesNotExistException || t instanceof UnsupportedAuthenticationException || t instanceof InvalidMessageException || t instanceof InvalidTopicNameException || t instanceof NotSupportedException || t instanceof NotAllowedException || t instanceof ChecksumException || t instanceof CryptoException || t instanceof ConsumerAssignException || t instanceof MessageAcknowledgeException || t instanceof TransactionConflictException || t instanceof ProducerBusyException || t instanceof ConsumerBusyException || t instanceof TransactionHasOperationFailedException) { return false; } return true; }

Solution

No response

Alternatives

No response

Anything else?

No response

Are you willing to submit a PR?

  • [X] I'm willing to submit a PR!

zifengmo avatar Jan 03 '24 08:01 zifengmo

@zifengmo Make sense. Could you push a PR to improve this behavior?

poorbarcode avatar Jan 29 '24 18:01 poorbarcode

I fixed it this way and re-packaged the jar, placing it in the lib directory for starting the proxy service. public static boolean isRetriableError(Throwable t) { if (t instanceof AuthorizationException || t instanceof InvalidServiceURL || t instanceof InvalidConfigurationException || t instanceof NotFoundException || t instanceof IncompatibleSchemaException || t instanceof TopicDoesNotExistException || t instanceof UnsupportedAuthenticationException || t instanceof InvalidMessageException || t instanceof InvalidTopicNameException || t instanceof NotSupportedException || t instanceof NotAllowedException || t instanceof ChecksumException || t instanceof CryptoException || t instanceof ConsumerAssignException || t instanceof MessageAcknowledgeException || t instanceof TransactionConflictException || t instanceof ProducerBusyException || t instanceof ConsumerBusyException || t instanceof TransactionHasOperationFailedException) { return false; } if (t.getMessage() != null && t.getMessage().contains("Subscription does not exist")) { return false; } return true; }

zifengmo avatar Feb 20 '24 05:02 zifengmo

@poorbarcode pr is here #22078

zifengmo avatar Feb 20 '24 07:02 zifengmo