fetchTopicOffsets throws error when destructuring

[talam]

I had a simple publisher/consumer setup running using a similar docker compose file setup to the one found in this repo and noticed that this code threw a destructing error.

        const high = await cluster.fetchTopicsOffset([
          {
            topic,
            fromBeginning: false,
            partitions: metadata.map(p => ({ partition: p.partitionId })),
          },
        ])

        const { partitions: highPartitions } = high.pop()

I don't have the exact error as i didn't copy it before my consumer's stdout log overwrote it in my terminal window due to limited length but it was something like

TypeError: Cannot destructure property `partitions` of 'undefined' or 'null'.

I suspect that cluster.fetchTopicsOffset is returning an empty array somehow causing the issue but I am unsure how that empty array would have been returned if my topic definitely has data in it. Even so, should there be a check added to that section to make sure we have something before trying to destructure it?

[maxim-slavutsky]

Hey, any progress on that issue? We have it happens very often while using fetchTopicOffsets method

[Nevon]

No one has been working on this, to my knowledge. If anyone who is affected by this could contribute a repro case that would be of massive help.

[samueltuckey]

I also run into this problem. It did not happen on older versions (1 year ago) of KafkaJs.

[everhardt]

I ran into this when doing multiple calls to kafka.admin().fetchTopicOffsets(topicName) in parallel. It turns out that metadata in https://github.com/tulios/kafkajs/blob/ff3b1117f316d527ae170b550bc0f772614338e9/src/admin/index.js#L310 is then sometimes [] in that case. I did not investigate why this is happening.