I get topic not available error.

[ffrankozz]

Hi Suat, Thank you for this useful code that implements CDC. However, I ran into a few issues when implementing it on my own. First off all,
1-I used the same docker-compose file as yours.

2-However, when creating the connectors, I needed to include a new line in the config files as it would refuse to create the connectors. What I included is the "topic.prefix" element because the I got the error saying that element was missing from the config file. I set the "topic.prefix" to "customer" and "identity" respectively for the two conffig files. That way I could create the connectors which I then made sure they existed. 3- I also noticed that when I ran the 3 apis, the migrations of the dbs did not take place so I manually ran "update-database" command to create the databases and their tables. 4- While running the apis, I had this exception about kafka topics which, however, did not stop api to proceed to be in the running state though. The error I was getting was about the topic names that we are registering in the startup classes. For instance, for the Customer api: The error reads like the following:

Confluent.Kafka.ConsumeException: Subscribed topic not available: user_events: Broker: Unknown topic or partition at Confluent.Kafka.Consumer2.Consume(Int32 millisecondsTimeout) at Shared.Kafka.Consumer.BackGroundKafkaConsumer2.ConsumeTopic(CancellationToken stoppingToken)

For the other apis, similar exceptions related to their topics registered in Kafka Consumer service. I wonder, if you could give me a clue as to the solution. It seems, the topics needed to be registered beforehand with Kafka. There is no automatic registration happening.

Thanks in advance.