pulsar-client-node icon indicating copy to clipboard operation
pulsar-client-node copied to clipboard

NodeJS Message router implementation

Open maximemg opened this issue 2 years ago • 3 comments

Hello,

My goal is to ensure a consumer will always receive key associated messages. But from my tests, on adding/removing consumers there are cases where Pulsar will do a rebalance.

So I think my best shot is to publish to a specific partition and subscribe to this specific partition with my customer.

So what I would do is implement the Message Router interface as explained here. But I could not make it in the Node.js client, as it does not seem to exist.

Will that be implemented in the future ?

Let me know if I'm mistaking or if it's unclear, as I'm pretty new to Apache Pulsar. Thanks for reading.

maximemg avatar Aug 07 '23 07:08 maximemg

Yes, CustomMessageRoute is not supported yet.

shibd avatar Aug 13 '23 05:08 shibd

I was wondering if there's a way to ensure messages will be received by the same consumer in a key shared sub ? Because so far I could not achieve that with keys.

Is there any ETA for the CustomMessageRoute ?

maximemg avatar Aug 16 '23 12:08 maximemg

I was wondering if there's a way to ensure messages will be received by the same consumer in a key shared sub ? Because so far I could not achieve that with keys.

  1. Consumers need to use KeyShared subscription type.
  2. and your producer need use KeyBased batchType (Node.js client is not yet implemented). Now you can disabled the producer batch first.
  // Create a producer and disabled batch
  const producer = await client.createProducer({
    topic: 'persistent://public/default/my-topic',
    sendTimeoutMs: 30000,
    batchingEnabled: false,    
  });
  // Create a consumer and use KeyShared sub type.
  const consumer = await client.subscribe({
    topic: 'persistent://public/default/my-topic',
    subscription: 'sub1',
    subscriptionType: 'KeyShared',
    ackTimeoutMs: 10000,
  });

shibd avatar Aug 16 '23 13:08 shibd