flink-cdc icon indicating copy to clipboard operation
flink-cdc copied to clipboard

Error serializing mysql8.0.18 field type Bigint data

Open xiaohui1227 opened this issue 2 years ago • 1 comments

Through the DataStream Source to listen to a table, use JsonDebeziumDeserializationSchema Field name order_id, type bigint(20) mysql order_id = 1518478400438665216 Flink order_id = "FRKx5vqAAAA =" Other values of the same type are fine

Environment :

  • Flink version : 1.13.3
  • Flink CDC version: 2.2.0
  • mysql version: 8.0.18

xiaohui1227 avatar Apr 25 '22 10:04 xiaohui1227

Problem is JsonDebeziumDeserializationSchema invoke org.apache.kafka.connect.json.JsonConverter which use org.apache.kafka.connect.data.Decimal as schemaName to convert mysql bigint unsigned. And by default JsonConverterConfig has DECIMAL_FORMAT_DEFAULT = DecimalFormat.BASE64.name(), so custom JsonConverterConfig and change DECIMAL_FORMAT_CONFIG to NUMERIC should fix the problem.

lumiseven avatar Jul 09 '22 05:07 lumiseven

Closing this issue because it was created before version 2.3.0 (2022-11-10). Please try the latest version of Flink CDC to see if the issue has been resolved. If the issue is still valid, kindly report it on Apache Jira under project Flink with component tag Flink CDC. Thank you!

PatrickRen avatar Feb 28 '24 15:02 PatrickRen