Mongo connectors null pointer exception

Open bobby-richard-sed opened this issue 3 years ago • 1 comments

Describe the bug(Please use English) Initial snapshot works fine. Eventually after the snapshot is finished the connector crashes with this exception: java.lang.NullPointerException at org.apache.flink.util.Preconditions.checkNotNull(Preconditions.java:59) at com.ververica.cdc.connectors.mongodb.table.MongoDBConnectorDeserializationSchema.deserialize(MongoDBConnectorDeserializationSchema.java:122) at com.ververica.cdc.debezium.internal.DebeziumChangeFetcher.handleBatch(DebeziumChangeFetcher.java:229) at com.ververica.cdc.debezium.internal.DebeziumChangeFetcher.runFetchLoop(DebeziumChangeFetcher.java:163) at com.ververica.cdc.debezium.DebeziumSourceFunction.run(DebeziumSourceFunction.java:446) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:67) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:323)

If I restart the connector and take a new snapshot, it completes the snapshot sucessfully but eventually I receive the same exception. It feels like the change stream is sending an event type that has a null _id. I would try filtering out event types that could contain null _id via the pipeline property, but that does not seem to be exposed in the flink connector.

Environment :

Flink version : 1.14.4
Flink CDC version: 2.2.1
Database and version: mongo 4.4

Jul 27 '22 18:07 bobby-richard-sed

Thanks @bobby-richard-sed to report this issue. Can you help provide a case to reproduce it?

Aug 02 '22 03:08 Jiabao-Sun