[Bug] [Module Name] Mysql database flows into Paimon field and is empty

[chen7652]

Search before asking

• [x] I had searched in the issues and found no similar issues.

What happened

使用dinky1.1、1.2的多表和单表的形式上，同步mysql数据库的数据流入到paimon时，会出现部分字段为空的情况

What you expected to happen

不清楚是不是和数据库有关系，有个数据库是正常同步的，但是这个数据库同步时就会出现空字段的情况。没有错误信息，数据会变更，但看不到大部分字段数据字段

How to reproduce

set 'table.exec.sink.upsert-materialize' = NONE; set 'execution.savepoint.ignore-unclaimed-state' = true; CREATE CATALOG xhtx WITH ( 'type' = 'paimon', 'warehouse' = '${paimon_warehouse}', 's3.endpoint' = '${paimon_host}', 's3.access-key' = '${paimon_access_key}', 's3.secret-key' = '${paimon_access_secret}', 'snapshot.time-retained' = '24h', 'snapshot.num-retained.min' = '100', 'snapshot.num-retained.max' = '99999999', 'snapshot.expire.execution-mode' = 'async', 'snapshot.expire.limit' = '100' );

use CATALOG xhtx; create database IF NOT EXISTS mysql; use mysql;

EXECUTE CDCSOURCE dinky_paimon_auto_create_table_wms WITH ( 'connector' = 'mysql-cdc', 'hostname' = '${wms_prod_host}', 'port' = '3306', 'username' = '${wms_prod_username}', 'password' = '${wms_prod_password}', 'checkpoint' = '10000', 'parallelism' = '1', -- 'scan.startup.mode' = 'initial', 'scan.startup.mode' = 'latest-offset', 'table-name' = 'wms.receipt_header_virtual', 'sink.connector' = 'paimon', 'sink.path' = 's3://xhtx-dev/paimon/mysql.db/#{tableName}', 'sink.table.prefix' = 'wms_' );

Anything else

No response

Version

1.2.1

Are you willing to submit PR?

• [x] Yes I am willing to submit a PR!

Code of Conduct

• [x] I agree to follow this project's Code of Conduct

[cxyuyu]

空的建表语句发下

[chen7652]

空的建表语句发下

CREATE TABLE wms_receipt_header_virtual ( id int(11) DEFAULT NULL, warehouseCode varchar(1048576) DEFAULT NULL, companyCode varchar(1048576) DEFAULT NULL, code varchar(1048576) DEFAULT NULL, trackSerialNum int(11) DEFAULT NULL, businessCode varchar(1048576) DEFAULT NULL, receiptType varchar(1048576) DEFAULT NULL, sourcePlatform varchar(1048576) DEFAULT NULL, sourceErp varchar(1048576) DEFAULT NULL, leadingSts int(11) DEFAULT NULL, trailingSts int(11) DEFAULT NULL, erpOrderCode varchar(1048576) DEFAULT NULL, erpOrderId varchar(1048576) DEFAULT NULL, erpOrderType varchar(1048576) DEFAULT NULL, purchaseOrderCode varchar(1048576) DEFAULT NULL, purchaseOrderId int(11) DEFAULT NULL, shipFromCode varchar(1048576) DEFAULT NULL, shipFromAddress1 varchar(1048576) DEFAULT NULL, shipFromAddress2 varchar(1048576) DEFAULT NULL, shipFromCity varchar(1048576) DEFAULT NULL, shipFromState varchar(1048576) DEFAULT NULL, shipFromDistrict varchar(1048576) DEFAULT NULL, shipFromCountry varchar(1048576) DEFAULT NULL, shipFromPostalCode varchar(1048576) DEFAULT NULL, shipFromName varchar(1048576) DEFAULT NULL, shipFromAttentionTo varchar(1048576) DEFAULT NULL, shipFromEmail varchar(1048576) DEFAULT NULL, shipFromPhoneNum varchar(1048576) DEFAULT NULL, shipFromMobile varchar(1048576) DEFAULT NULL, shipFromFaxNum varchar(1048576) DEFAULT NULL, scheduledArriveDate datetime DEFAULT NULL, actualArriveDateTime datetime DEFAULT NULL, recvDock varchar(1048576) DEFAULT NULL, closedAt datetime DEFAULT NULL, closedBy varchar(1048576) DEFAULT NULL, startCheckinDatetime datetime DEFAULT NULL, endCheckinDatetime datetime DEFAULT NULL, qtyRatio int(11) DEFAULT NULL, totalQty int(11) DEFAULT NULL, totalLines int(11) DEFAULT NULL, totalWeight decimal(15, 4) DEFAULT NULL, totalVolume decimal(15, 4) DEFAULT NULL, uploadRequired int(11) DEFAULT NULL, hostCompanyCode varchar(1048576) DEFAULT NULL, receiptNote varchar(1048576) DEFAULT NULL, shippingNote varchar(1048576) DEFAULT NULL, locked int(11) DEFAULT NULL, lockedBy varchar(1048576) DEFAULT NULL, holdCode varchar(1048576) DEFAULT NULL, created datetime DEFAULT NULL, createdBy varchar(1048576) DEFAULT NULL, lastUpdated datetime DEFAULT NULL, lastUpdatedBy varchar(1048576) DEFAULT NULL, version int(11) DEFAULT NULL, userDef1 varchar(1048576) DEFAULT NULL, userDef2 varchar(1048576) DEFAULT NULL, userDef3 varchar(1048576) DEFAULT NULL, userDef4 varchar(1048576) DEFAULT NULL, userDef5 varchar(1048576) DEFAULT NULL, userDef6 varchar(1048576) DEFAULT NULL, userDef7 varchar(1048576) DEFAULT NULL, userDef8 varchar(1048576) DEFAULT NULL, processStamp varchar(1048576) DEFAULT NULL, uploadBatch varchar(1048576) DEFAULT NULL, shipmentId int(11) DEFAULT NULL, returnStatus varchar(1048576) DEFAULT NULL, returnWaybillCode varchar(1048576) DEFAULT NULL, videoFileName varchar(1048576) DEFAULT NULL, postReceiptOn date DEFAULT NULL, blind int(11) DEFAULT NULL, carrierCode varchar(1048576) DEFAULT NULL, consolidateCode varchar(1048576) DEFAULT NULL, warehouseTransferId int(11) DEFAULT NULL, warehouseTransferCode varchar(1048576) DEFAULT NULL, auditStatus int(11) DEFAULT NULL, auditedBy varchar(1048576) DEFAULT NULL, auditedAt datetime DEFAULT NULL, auditOpinion varchar(1048576) DEFAULT NULL, tempLevel varchar(1048576) DEFAULT NULL, checkBusinessScope int(11) DEFAULT NULL, arrivalRegistration int(11) DEFAULT NULL, arrivalRegistrationRuleCode varchar(1048576) DEFAULT NULL, iqcStatus int(11) DEFAULT NULL, iqcPoint int(11) DEFAULT NULL, shipFromCode2 varchar(1048576) DEFAULT NULL, shipFromName2 varchar(1048576) DEFAULT NULL, recvObject int(11) DEFAULT NULL, erpNumber varchar(1048576) DEFAULT NULL, sourceERPCompanyCode varchar(1048576) DEFAULT NULL, sourceERPWarehouseCode varchar(1048576) DEFAULT NULL ) PROPERTIES ("location" = "s3://xhtx-dev/paimon/mysql.db/wms_receipt_header_virtual");

[chen7652]

空的建表语句发下

初步排查发现源表存在驼峰的字段数据，但是小写字段数据可以正常流入，想问问有什么配置参数吗

[lianchaoqi]

空的建表语句发布

初步排查发现源表存在驼峰的字段数据，但是小写字段数据可以正常流入，想询问有什么配置参数吗？

我的非驼峰字段也会出现这种情况，其中重要的发现就是字段类型为datetime

[github-actions[bot]]

Hello @, this issue has not been active for more than 30 days. This issue will be closed in 7 days if there is no response. If you have any questions, you can comment and reply.

你好 @, 这个 issue 30 天内没有活跃，7 天后将关闭，如需回复，可以评论回复。

[kevinlin299]

空的建表语句发布

初步排查发现源表存在驼峰的字段数据，但是小写字段数据可以正常流入，想询问有什么配置参数吗？

我的非驼峰字段也会出现这种情况，其中重要的发现就是字段类型为datetime

应该是涉及字段名为大写的都会出现这个情况，无论是cdc-source还是正常的flink-sql都是