apache
[FLINK-35391][cdc-connector][paimon] Bump dependency of Paimon Pipeline connector to 0.8.0
@lvyanquan cc, thanks. Paimon 0.8.0 use dynamic bucket as default, and FLINK-35394 prepare to support dynamic bucket. For bump paimon to 0.8.0, we can set unit test paimon table bucket default 1 first.
PS: An error will be reported during local testing with a lower version of jdk (like jdk 1.8.0). Does it need to be fixed?
[ERROR] Error: VM option 'UseG1GC' is experimental and must be enabled via -XX:+UnlockExperimentalVMOptions.
PS: An error will be reported during local testing with a lower version of jdk (like jdk 1.8.0). Does it need to be fixed?
[ERROR] Error: VM option 'UseG1GC' is experimental and must be enabled via -XX:+UnlockExperimentalVMOptions.
Does this error only affect local testing and does not affect running with jar? If it does not affect the operation after packaging, it can be ignored.
Does this error only affect local testing and does not affect running with jar? If it does not affect the operation after packaging, it can be ignored.
OK, just affect local testing.
Considering this optimization, upgrading the version is necessary. https://github.com/apache/paimon/pull/2943
Paimon team has released 8.0.1 patch version last month at here. Maybe we can upgrade to this version without encountering any compatibility issues.
Could @leonardBang help driving this? Just noticed that Paimon 0.8.2 got released...
@beryllw Would you like to bump paimon version to 8.0.2?
done(paimon 0.8.2)
Exception in MySQL CDC test.
2024-07-16T10:37:08.5104004Z 52150 [debezium-reader-1] INFO org.apache.flink.cdc.connectors.mysql.debezium.task.MySqlSnapshotSplitReadTask - For split 'date_convert_test_370b26.date_convert_test:0' of table date_convert_test_370b26.date_convert_test using select statement: 'SELECT * FROM `date_convert_test_370b26`.`date_convert_test`'
2024-07-16T10:37:08.5109144Z 52153 [debezium-reader-1] ERROR io.debezium.relational.TableSchemaBuilder - Failed to properly convert data value for 'date_convert_test_370b26.date_convert_test.test_time' of type TIME for row [1, 2023-04-01T14:23:00.000+0000, 2023-04-01T14:24:00.000+0000, 2023-04-01, PT14H25M]:
2024-07-16T10:37:08.5111918Z java.time.temporal.UnsupportedTemporalTypeException: Unsupported field: YearOfEra
2024-07-16T10:37:08.5113137Z at java.time.LocalTime.get0(LocalTime.java:679) ~[?:1.8.0_412]
2024-07-16T10:37:08.5114369Z at java.time.LocalTime.getLong(LocalTime.java:656) ~[?:1.8.0_412]
2024-07-16T10:37:08.5115694Z at java.time.format.DateTimePrintContext.getValue(DateTimePrintContext.java:298) ~[?:1.8.0_412]
2024-07-16T10:37:08.5117433Z at java.time.format.DateTimeFormatterBuilder$NumberPrinterParser.format(DateTimeFormatterBuilder.java:2551) ~[?:1.8.0_412]
2024-07-16T10:37:08.5119341Z at java.time.format.DateTimeFormatterBuilder$CompositePrinterParser.format(DateTimeFormatterBuilder.java:2190) ~[?:1.8.0_412]
2024-07-16T10:37:08.5121425Z at java.time.format.DateTimeFormatter.formatTo(DateTimeFormatter.java:1746) ~[?:1.8.0_412]
2024-07-16T10:37:08.5124208Z at java.time.format.DateTimeFormatter.format(DateTimeFormatter.java:1720) ~[?:1.8.0_412]
2024-07-16T10:37:08.5125482Z at java.time.LocalTime.format(LocalTime.java:1413) ~[?:1.8.0_412]
2024-07-16T10:37:08.5127020Z at io.debezium.connector.mysql.converters.MysqlDebeziumTimeConverter.formatTime(MysqlDebeziumTimeConverter.java:266) ~[classes/:?]
2024-07-16T10:37:08.5128916Z at io.debezium.connector.mysql.converters.MysqlDebeziumTimeConverter.convertToTime(MysqlDebeziumTimeConverter.java:240) ~[classes/:?]
2024-07-16T10:37:08.5131335Z at io.debezium.connector.mysql.converters.MysqlDebeziumTimeConverter.lambda$registerDateConverter$1(MysqlDebeziumTimeConverter.java:140) ~[classes/:?]
2024-07-16T10:37:08.5134109Z at io.debezium.relational.CustomConverterRegistry.lambda$getValueConverter$0(CustomConverterRegistry.java:150) ~[debezium-core-1.9.8.Final.jar:3.2-SNAPSHOT]
2024-07-16T10:37:08.5136466Z at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$5(TableSchemaBuilder.java:297) ~[debezium-core-1.9.8.Final.jar:3.2-SNAPSHOT]
Is Timeout the root cause of failure？And log.
2024-07-16T10:37:08.8936449Z [INFO] Results:
2024-07-16T10:37:08.8936866Z [INFO]
2024-07-16T10:37:08.8937259Z [ERROR] Errors:
2024-07-16T10:37:08.8939761Z [ERROR] NewlyAddedTableITCase.testNewlyAddedTableForExistsPipelineThrice:193->testNewlyAddedTableOneByOne:724->testNewlyAddedTableOneByOne:861->MySqlSourceTestBase.waitForUpsertSinkSize:175 » TestTimedOut
2024-07-16T10:37:08.8941643Z [INFO]
2024-07-16T10:37:08.8942175Z [ERROR] Tests run: 194, Failures: 0, Errors: 1, Skipped: 0
@czy006 PTAL as this failed case maybe related to https://github.com/apache/flink-cdc/pull/3240.
@leonardBang It seems that the failure of unit ci has nothing to do with Paimon's changes.