help with consumer not processing new message intermittently

[sayan-mitra]

hi

I'm observing a behaviour with few of my topics, where few of my topic consumers are not picking new messages. I debugged around and following is the observation :

1. readloop is reading the the offset, read loop.

2. ReadMessage is getting the messages, batch read message

3. sendMessage happens without error and offset gets incremented, send message

4. steps 1-3 occurs with new messages in the partition and offset gets incremented properly.

5. FetchMessage called by application code and this runs for a few messages successfully, fetch message

6. Message is committed by application code and log for successful commit comes commit log.

7. calling 5 again, then hangs and doesn't fetch new messages.

there are no error logs from the library to suggest if something went wrong.

Can someone please suggest what can be going wrong here or any pointers to check around.

Thanks.

[achille-roussel]

Hello @sayan-mitra, thanks for reporting.

Do you happen to know if your issue might be the same as the one reported in https://github.com/segmentio/kafka-go/issues/955 ?

[achille-roussel]

As follow up here, would it be possible for you to provide a code snippet that reproduces the issue, or stack traces collected from your application when the problem occurs?

[achille-roussel]

I'm going to close this due to inactivity, feel free to reopen if it requires further discussion!