fix: refreshMetadata should retry if topic does not exist and allowAu…

[regevbr]

…toTopicCreation is set

fix: #815

Not sure why it happens but it happens randomly. Seems like a race condition when 2 client try to create the same topic at the same time. This fix is tested and working

[hoke-t]

will this be merged soon?

[atiquefiroz]

When will this be fixed and merged? @Nevon

[caio-favero]

Any idea when this will be merged?

[florian-besser]

This PR looks promising in fixing part (or all of) of https://github.com/tulios/kafkajs/issues/815

Basically ever since Kafka moved from Zookeeper to KRaft there seems to have been a breaking change in auto-topic creation on subscription. It looks like with zookeeper back in the day you could subscribe to a non-existent topic just fine, it would be created behind the scenes and the subscription succeeds. now with KRaft subscribing to a non-existent topic throws KafkaJSProtocolError: This server does not host this topic-partition but still creates the topic afterwards, if auto-topic-creation is enabled.

With this PR the (correct) action of retrying the subscription after waiting a bit seems to be used.

As this PR does not have any reviewers maybe you could have a look @tulios ?

[florian-besser]

The PR would also close https://github.com/tulios/kafkajs/issues/1563

[HeatherFlux]

This looks good to go to me.

[Arman92]

Any updates on this @Nevon @tulios, can this be merged?

[leppaott]

This would be nice to have in a new release? :pray: @Nevon @tulios