Ling Jin

At the very first, the environment looks like this: ``` ALLOW_PLAINTEXT_LISTENER: yes KAFKA_CFG_OFFSETS_TOPIC_REPLICATION_FACTOR: "3" KAFKA_CFG_TRANSACTION_STATE_LOG_MIN_ISR: "2" KAFKA_CFG_ZOOKEEPER_CONNECT: downstream-zk.${testbed}:2181 KAFKA_HEAP_OPTS: -Xmx512M -Xms512M -XX:ParallelGCThreads=1 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: "3" KAFKA_OPTS: -Dlogging.level=INFO KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: "2" KAFKA_ZOOKEEPER_CONNECT: downstream-zk.${testbed}:2181...

This is fixed by set environment variable, close this issue: ``` KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: "PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT" ```

The minimum reproduce docker compose file looks like this: ``` version: '3.7' services: zookeeper: image: zookeeper container_name: zookeeper ports: - "2181:2181" volumes: - /tmp/zookeeper/data:/data - /tmp/zookeeper/datalog:/datalog - /tmp/zookeeper/logs:/logs restart: always...

> This is fixed by set environment variable, close this issue: > > ``` > KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: "PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT" > ``` I am curious why I have to set this environment variable...

/retest

Pause and remove changefeed, close the kv client, this should also close the gRPC connections to the TiKV-CDC, and cancel connections.

I am trying to reproduce this scenario on my local test environment, here is some observations: * There are more than 100,000 regions in the TiKV cluster. On the TiCDC...

* The changefeed paused around 09:40 * The event feed count is not dropped to 0 immediately * KV Client still receive a lot of kv entries

``` [2024/10/02 01:38:47.996 +00:00] [INFO] [feed_state_manager.go:243] ["handle admin job"] [namespace=default] [changefeed=changefeed-1841264655229501440] [job="{\"CfID\":{\"Namespace\":\"default\",\"ID\":\"changefeed-1841264655229501440\"},\"Type\":1,\"Error\":null,\"OverwriteCheckpointTs\":0}"] [2024/10/02 01:38:47.996 +00:00] [INFO] [ddl_puller.go:692] ["close the ddl puller"] [namespace=default] [changefeed=changefeed-1841264655229501440_ddl_puller] [2024/10/02 01:38:48.091 +00:00] [INFO] [changefeed.go:723] ["changefeed closed"]...

``` [2024/10/01 23:58:50.650 +00:00] [INFO] [processor.go:1242] ["Processor try to close source manager"] [namespace=default] [changefeed=changefeed-1841264655229501440] [2024/10/01 23:58:50.650 +00:00] [INFO] [manager.go:132] ["Closing source manager"] [namespace=default] [changefeed=changefeed-1841264655229501440] ... [2024/10/02 00:20:13.166 +00:00] [INFO] [manager.go:140]...