node-rdkafka
node-rdkafka copied to clipboard
Increase in topic partition errors out with PARTCNT and severity = 5
If the number of partitions on a given topic is increased, ideally the consumer should register automatically for those partitions and start consuming the events. In our case, the consumer started consuming the events 3 days after the new partitions were added. Old events were not fetched too. Can you please provide insights on what is that config which needs to be done to automatically register the newly added partitions? We are using event.log to log the events. This is what we got in our logs :
{
"severity": 5,
"fac": "PARTCNT",
"context": {}
}
Environment Information
- OS [e.g. Mac, Arch, Windows 10]: Alpine Linux v3.11
- Node Version [e.g. 8.2.1]: 10.24.1
- NPM Version [e.g. 5.4.2]: 6.14.12
- C++ Toolchain [e.g. Visual Studio, llvm, g++]:
- node-rdkafka version [e.g. 2.3.3]: 2.6.1
Steps to Reproduce
node-rdkafka Configuration Settings
'group.id': "test",
'enable.auto.commit': false,
'fetch.message.max.bytes': 10048576,
'offset_commit_cb': (error: any) => {
if (!error) return;
},
'statistics.interval.ms': 1000,
Additional context