emqx-web-hook icon indicating copy to clipboard operation
emqx-web-hook copied to clipboard

Published 20 hours ago • verified publisher icon emqx

emqx-web-hook is blocking message being received by subscribers

Open daadu opened this issue 4 years ago • 9 comments

Setup

version: 4.1.0 environment: k8s (installed with Helm) mqtt: v3.1

Problem

The web-hook call made by emqx seems like a blocking call for subscribers from receiving "that" message. Until the web hook is processed the messages are not received by subscriber. This causes an issue particularly when the webserver receiving a webhook is also (temporarily) subscribing to a particular topic (in my case to perform request-response pattern messages when a certain message is received via webhook).

Possible Solution

  1. trigger webhook after messages are routed to all subscribers.
  2. fire-and-forget (async) webhook should be trigger (I posted regarding this in #196)

Remarks

If 4.2.0 is not happening soon then solution suggested in point 1 should be quickly implemented as a work around. Also will this issue also be raised when using emx-rule-engine instead of emqx-web-hook?

daadu avatar Jul 27 '20 07:07 daadu

Any updates on this?

We are running 4.2.6 and having still this problem

hmonadjem avatar Jan 19 '21 18:01 hmonadjem

@hmonadjem can you use message queue to process webhook message , so that it won't block the whole process ?

KerryJava avatar Jan 21 '21 11:01 KerryJava

@hmonadjem can you use message queue to process webhook message , so that it won't block the whole process ?

Already tried it, used "celery" for it, it causes hick-ups when doing request-response kind of a thing on the HTTP server.

daadu avatar Jan 21 '21 12:01 daadu

Hi, guys, This has really been a problem with the web-hook plugin.

We will put this on the agenda in anticipation of a good solution. Thank you very much for your suggestions and concerns!

HJianBo avatar Jan 26 '21 06:01 HJianBo

Have you guys tested the approximate throughput and TPS that can be supported by the current version?

HJianBo avatar Jan 26 '21 06:01 HJianBo

Thank you! The Throughput was never a problem for us (we have about 20 msg/s) but it is a problem when the target http server is booting up or shutting down or is unreachable. That basically blocks every message for 5s (timeout) and after that if the response is 500 the mqtt message is never delivered. Which is very bad for us.

hmonadjem avatar Jan 26 '21 07:01 hmonadjem

Is there any progress on this? I'm running on the same issue.

adrian-parisi avatar Jun 10 '21 21:06 adrian-parisi

I hope this gets resolved, because it's really bad, where message_publish and message_delivered don't work as they should.

onyet avatar Sep 09 '21 09:09 onyet

Hello, Is there any progress on this?

QBH-insist avatar Feb 22 '22 09:02 QBH-insist