nodejs-pubsub
nodejs-pubsub copied to clipboard
googleapis
Examples of enterprise grade implementations with this library?
I am trying to build an app in Kubernetes using the nestjs framework. There are multiple pods pulling from the same subscription. However, when I run it at full production message volumes, I immediately get a backlog of unacked messages, while some pods sit idle, receiving no messages. When running the same app using a different messaging provider, we have no trouble keeping up with the volume. The problem does not lie in the app logic of processing message data.
I just don't know where to turn to solve these problems. If someone could point me towards some example usages of this library to build a enterprise grade, high-volume app, I would love to see it. The Pub/Sub service itself is something of a black box, and I've found the documentation and examples for this library to be a bit thin.
I have already tried to vary the flow control max messages. I have even gone so far as to ack every message immediately upon delivery to ensure that no uncaught errors in processing will leave a message unacked. Even so, all processing is in try/catch and no errors are logged.
See also https://stackoverflow.com/questions/73533498/google-pub-sub-with-distributed-subscribers-in-node-js
@larsenqec Yeah, this sort of understanding is a really valid question, I think. I was trying to start an effort to make more of these bigger end-to-end examples so that it's not just "here's how you call this one API". There's a group now that is tasked with doing exactly that, so I'll send this over there to see if anyone has thoughts.
@kamalaboulhosn Maybe you have some insight on the behaviour of the multiple subscribers?
The Pub/Sub service is designed to scale to gigabytes per second, so it should be able to handle a full production message volume. We have a blog post about the expected capacity of a single instance of the clients across languages and I'd expect it to scale horizontally with more clients.
@larsenqec your best step is probably to put in a support ticket with the details of your project and subscription and the times at which your workload was running so that backend engineers can better investigate why the load is not being consumed.
So it turned out to be a problem with message ordering implementation on our end, but I am still frustrated with the documentation.
https://stackoverflow.com/a/73655086/2917389