faust icon indicating copy to clipboard operation
faust copied to clipboard

Consumer doesn't consume from some of its assigned partition

Open s34vv1nd opened this issue 2 years ago • 1 comments

Checklist

  • [x] I have included information about relevant versions
  • [ ] I have verified that the issue persists when using the master branch of Faust.

Steps to reproduce

I have 3 faust workers of the same consumer group (same app's id). The group consume from multiple topics, one of them has 12 partitions. Each worker is correctly assigned 4 partitions.

Expected behavior

Each worker can consume messages from all 4 assigned partitions

Actual behavior

Only one worker can consume from all 4 partition, the other two workers are still running but do not consume from all of their partitions

image

image

Versions

  • Python version: 3.10.10
  • Faust streaming version: 0.8.11
  • Operating system: Debian GNU/Linux 11
  • Kafka version: 3.2.1

s34vv1nd avatar Mar 17 '23 03:03 s34vv1nd

What processing logic is done for each message? Is it expensive and possibly blocking the event loop?

szicari-streambit avatar Aug 15 '23 03:08 szicari-streambit