flink-cdc
flink-cdc copied to clipboard
tidb cdc synchronous error when the amount of data is millions
Describe the bug(Please use English) A clear and concise description of what the bug is. Synchronous error when the amount of data is millions, the same table data volume of 10000 will not report an error
Environment :
- Flink version : 1.15.1
- Flink CDC version: flink-sql-connector-tidb-cdc-2.2.1.jar
- Database and version: tidb 6.1
To Reproduce Steps to reproduce the behavior:
- The test data :
- The test code :
- The error : 2022-08-18 11:19:34,325 WARN org.tikv.common.region.AbstractRegionStoreClient [] - no followers of region[4052] available, retry 2022-08-18 11:19:34,326 WARN org.tikv.common.operation.RegionErrorHandler [] - request failed because of: UNKNOWN 2022-08-18 11:19:34,333 INFO org.tikv.cdc.CDCClient [] - remove regions: [] 2022-08-18 11:19:34,333 WARN org.apache.flink.runtime.taskmanager.Task [] - Source: cnft_item_backup[1] (1/1)#121 (c40c3c7e9651b3343fa3c8f33eadd159) switched from RUNNING to FAILED with failure cause: org.tikv.common.exception.TiClientInternalException: Error scanning data from region. at org.tikv.common.operation.iterator.ScanIterator.cacheLoadFails(ScanIterator.java:115) at org.tikv.common.operation.iterator.ConcreteScanIterator.hasNext(ConcreteScanIterator.java:105) at java.base/java.util.Iterator.forEachRemaining(Unknown Source) at org.tikv.txn.KVClient.scan(KVClient.java:117) at com.ververica.cdc.connectors.tidb.TiKVRichParallelSourceFunction.readSnapshotEvents(TiKVRichParallelSourceFunction.java:172) at com.ververica.cdc.connectors.tidb.TiKVRichParallelSourceFunction.run(TiKVRichParallelSourceFunction.java:127) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:67) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:332) Caused by: org.tikv.common.exception.TiClientInternalException: ScanResponse failed without a cause at org.tikv.common.region.RegionStoreClient.isScanSuccess(RegionStoreClient.java:315) at org.tikv.common.region.RegionStoreClient.scan(RegionStoreClient.java:306) at org.tikv.common.region.RegionStoreClient.scan(RegionStoreClient.java:346) at org.tikv.common.operation.iterator.ConcreteScanIterator.loadCurrentRegionToCache(ConcreteScanIterator.java:80) at org.tikv.common.operation.iterator.ScanIterator.cacheLoadFails(ScanIterator.java:80) ... 8 more
Additional Description If applicable, add screenshots to help explain your problem.