[CDCSDK] gRPC Connector going OOM in without tablet splitting master CDC case on latest 1.9.5.y.220.4 (Not seen on 1.9.5.220.2)

[shamanthchandra-yb]

Jira Link: DB-12195

Description

Testcase name: test_cdc_comply_advantage_without_tablet_splitting

Initially thought of data loss issue, but later discovered that its because of java.lang.OutOfMemoryError: Java heap space issue.

Did many experiments with various configurations (Please find in JIRA comments) on server 2.20, 2024.1 and master, with 220.2 and 220.4 connector. Issue is only seen with 220.4 connector.

Source connector version

1.9.5.y.220.4

Connector configuration

add connector connector_name='ybconnector_cdc_ac1869_test_cdc_cc158a' stream_id='bc3adc8fcb160fade24d7edcd12c2a77' db_name='cdc_ac1869' connector_host='172.151.19.26' table_list=['test_cdc_cc158a'] {'name': 'ybconnector_cdc_ac1869_test_cdc_cc158a', 'config': {'database.master.addresses': '172.151.30.246:7100,172.151.18.155:7100,172.151.30.28:7100', 'database.hostname': '172.151.30.246:5433,172.151.18.155:5433,172.151.30.28:5433', 'database.port': 5433, 'database.masterhost': '172.151.30.246', 'database.masterport': '7100', 'database.user': 'yugabyte', 'database.password': 'yugabyte', 'database.dbname': 'cdc_ac1869', 'snapshot.mode': 'initial', 'admin.operation.timeout.ms': 600000, 'socket.read.timeout.ms': 300000, 'max.connector.retries': '10', 'operation.timeout.ms': 600000, 'topic.creation.default.compression.type': 'lz4', 'topic.creation.default.cleanup.policy': 'delete', 'topic.creation.default.partitions': 2, 'topic.creation.default.replication.factor': '1', 'tasks.max': '5', 'connector.class': 'io.debezium.connector.yugabytedb.YugabyteDBConnector', 'database.server.name': 'db_cdc', 'database.streamid': 'bc3adc8fcb160fade24d7edcd12c2a77', 'table.include.list': 'public.test_cdc_cc158a', 'database.sslrootcert': '/kafka/ca.crt'}}

YugabyteDB version

2.23.0.0-b625

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

• [X] I confirm this issue does not contain any sensitive information.