Investigate issues with Kafka<>QuestDB

[joanagmaia]

Issue 1

activity with id = a3f3dbbf-960a-4d3a-92b9-47befea3efef is present in:

• activityRelations
• KAFKA_INGEST.COMMUNITY_MANAGEMENT.ACTIVITIES

However, it does not exist in QuestDB. Because of this it also didn't flow to postgres and Fivetran.

Assumption: It seems that the activity reaches the Kafka topic which is then consumed by Snowflake Kafka Connector, but for some reason the Kafka QuestDB connector did not work.

Issue 2

https://cloud.temporal.io/namespaces/cm-oracle-prod.kbdru/workflows/finishMemberMerging%2Fdeac53bf-6fad-4aca-9747-ae0784542c95%2Facda51d0-e110-11ef-ac21-a5afb6101e68/0195813a-95d8-786a-8553-999eec4fd124/history

We can see in Temporal that the merge was successful and nothing appears to have gone wrong. However, checking QuestDB we still have activities associated to both members even after the second one was deleted.

select * from activities where memberId = 'deac53bf-6fad-4aca-9747-ae0784542c95';
select * from activities where memberId = 'acda51d0-e110-11ef-ac21-a5afb6101e68';

• activityRelations seems to be correct, with no activities for the secondary member.

Assumption: Again it seems that something gets lost in the Kafka QuestDB connector.

Issue 3

The Kafka connect questDB sync gets stuck sometimes and has to be manually restarted. We should try to find the root cause for this, or have more visibility on the errors to be able to act on this.