clickhouse-java icon indicating copy to clipboard operation
clickhouse-java copied to clipboard
verified publisher icon ClickHouse

Couldn't use clickhouse-jdbc 0.8.x with liquibase

Open eldarst opened this issue 7 months ago • 1 comments

Description

I'm using liquibase version 4.32.0, when liquibase trying to parse clickhouse column meta data, it falls when parsing DATA_TYPE field. It expected to be Integer, and in 0.7.x versions it used to return as Int, but in 0.8.x version it just return String.

Steps to reproduce

  1. Connect to Clickhouse using clickhouse-jdbc 0.8.x
  2. Instantiate new Liquibase
  3. liquibase.update

Error Log or Exception StackTrace

[main] INFO com.clickhouse.jdbc.ClickHouseDriver - ClickHouse JDBC driver version: clickhouse-jdbc 0.8.6 (revision: 2d305b7) [main] INFO com.clickhouse.jdbc.ClickHouseDriver - v2 driver [main] INFO com.clickhouse.jdbc.Driver - ClickHouse JDBC driver version: 0.8.6 [main] INFO com.clickhouse.jdbc.ClickHouseDriver - ClickHouse JDBC driver version: clickhouse-jdbc 0.8.6 (revision: 2d305b7) [main] INFO com.clickhouse.jdbc.ClickHouseDriver - v2 driver [main] INFO com.clickhouse.jdbc.ClickHouseDriver - v2 driver [main] INFO com.clickhouse.jdbc.metadata.DatabaseMetaDataImpl - getTables: catalog={}, schemaPattern={}, tableNamePattern={}, types={} мая 29, 2025 4:21:48 PM liquibase.command ERROR: Exception Details ERROR: Exception Details

INFO: Logging exception. ERROR: Exception Primary Class: SQLException ERROR: Exception Primary Reason: Failed to convert String to java.lang.Integer ERROR: Exception Primary Source: ClickHouse 25.5.1.2782 мая 29, 2025 4:21:48 PM liquibase.command INFO: Command execution complete ERROR: Exception Primary Class: SQLException

ERROR: Exception Primary Reason: Failed to convert String to java.lang.Integer

ERROR: Exception Primary Source: ClickHouse 25.5.1.2782

Exception in thread "main" liquibase.exception.CommandExecutionException: liquibase.exception.ChangeLogParseException: liquibase.exception.UnexpectedLiquibaseException: liquibase.exception.DatabaseException: java.sql.SQLException: Failed to convert String to java.lang.Integer at liquibase.command.CommandScope.lambda$execute$6(CommandScope.java:310) at liquibase.Scope.child(Scope.java:225) at liquibase.Scope.child(Scope.java:201) at liquibase.command.CommandScope.execute(CommandScope.java:251) at liquibase.Liquibase.lambda$update$0(Liquibase.java:216) at liquibase.Scope.lambda$child$0(Scope.java:216) at liquibase.Scope.child(Scope.java:225) at liquibase.Scope.child(Scope.java:215) at liquibase.Scope.child(Scope.java:194) at liquibase.Liquibase.runInScope(Liquibase.java:1366) at liquibase.Liquibase.update(Liquibase.java:205) at liquibase.Liquibase.update(Liquibase.java:188) at org.example.MigrationService.migrate(MigrationService.java:38) at org.example.Main.main(Main.java:36) Caused by: liquibase.exception.ChangeLogParseException: liquibase.exception.UnexpectedLiquibaseException: liquibase.exception.DatabaseException: java.sql.SQLException: Failed to convert String to java.lang.Integer Caused by: liquibase.exception.ChangeLogParseException: liquibase.exception.UnexpectedLiquibaseException: liquibase.exception.DatabaseException: java.sql.SQLException: Failed to convert String to java.lang.Integer

at liquibase.parser.core.xml.AbstractChangeLogParser.parse(AbstractChangeLogParser.java:25)
at liquibase.command.core.helpers.DatabaseChangelogCommandStep.lambda$getDatabaseChangeLog$0(DatabaseChangelogCommandStep.java:141)
at liquibase.Scope.child(Scope.java:225)
at liquibase.Scope.child(Scope.java:201)
at liquibase.command.core.helpers.DatabaseChangelogCommandStep.getDatabaseChangeLog(DatabaseChangelogCommandStep.java:140)
at liquibase.command.core.helpers.DatabaseChangelogCommandStep.run(DatabaseChangelogCommandStep.java:83)
at liquibase.command.CommandScope.lambda$execute$6(CommandScope.java:263)
... 13 more

Caused by: liquibase.exception.UnexpectedLiquibaseException: liquibase.exception.DatabaseException: java.sql.SQLException: Failed to convert String to java.lang.Integer Caused by: liquibase.exception.UnexpectedLiquibaseException: liquibase.exception.DatabaseException: java.sql.SQLException: Failed to convert String to java.lang.Integer

at liquibase.changelog.StandardChangeLogHistoryService.init(StandardChangeLogHistoryService.java:115)
at liquibase.changelog.DatabaseChangeLog.include(DatabaseChangeLog.java:1125)
at liquibase.changelog.DatabaseChangeLog.handleInclude(DatabaseChangeLog.java:646)
at liquibase.changelog.DatabaseChangeLog.handleChildNodeHelper(DatabaseChangeLog.java:479)
at liquibase.changelog.DatabaseChangeLog.handleChildNode(DatabaseChangeLog.java:464)
at liquibase.changelog.DatabaseChangeLog.load(DatabaseChangeLog.java:431)
at liquibase.parser.core.xml.AbstractChangeLogParser.parse(AbstractChangeLogParser.java:23)
... 19 more

Caused by: liquibase.exception.DatabaseException: java.sql.SQLException: Failed to convert String to java.lang.Integer Caused by: liquibase.exception.DatabaseException: java.sql.SQLException: Failed to convert String to java.lang.Integer

at liquibase.snapshot.ResultSetCache.get(ResultSetCache.java:100)
at liquibase.snapshot.JdbcDatabaseSnapshot$CachingDatabaseMetaData.getColumns(JdbcDatabaseSnapshot.java:396)
at liquibase.snapshot.jvm.ColumnSnapshotGenerator.addTo(ColumnSnapshotGenerator.java:167)
at liquibase.snapshot.jvm.JdbcSnapshotGenerator.snapshot(JdbcSnapshotGenerator.java:76)
at liquibase.snapshot.SnapshotGeneratorChain.snapshot(SnapshotGeneratorChain.java:71)
at liquibase.snapshot.DatabaseSnapshot.include(DatabaseSnapshot.java:319)
at liquibase.snapshot.DatabaseSnapshot.init(DatabaseSnapshot.java:112)
at liquibase.snapshot.DatabaseSnapshot.<init>(DatabaseSnapshot.java:65)
at liquibase.snapshot.JdbcDatabaseSnapshot.<init>(JdbcDatabaseSnapshot.java:37)
at liquibase.snapshot.SnapshotGeneratorFactory.createSnapshot(SnapshotGeneratorFactory.java:343)
at liquibase.snapshot.SnapshotGeneratorFactory.createSnapshot(SnapshotGeneratorFactory.java:372)
at liquibase.snapshot.SnapshotGeneratorFactory.getDatabaseChangeLogTable(SnapshotGeneratorFactory.java:383)
at liquibase.changelog.StandardChangeLogHistoryService.init(StandardChangeLogHistoryService.java:112)
... 25 more

Caused by: java.sql.SQLException: Failed to convert String to java.lang.Integer at com.clickhouse.jdbc.internal.JdbcUtils.convert(JdbcUtils.java:195) at com.clickhouse.jdbc.ResultSetImpl.getObject(ResultSetImpl.java:1523) Caused by: java.sql.SQLException: Failed to convert String to java.lang.Integer

at com.clickhouse.jdbc.ResultSetImpl.getObject(ResultSetImpl.java:460)
at liquibase.util.JdbcUtil.getResultSetValue(JdbcUtil.java:81)
at liquibase.executor.jvm.ColumnMapRowMapper.getColumnValue(ColumnMapRowMapper.java:81)
at liquibase.snapshot.ResultSetCache$ResultSetExtractor$1.getColumnValue(ResultSetCache.java:294)
at liquibase.executor.jvm.ColumnMapRowMapper.mapRow(ColumnMapRowMapper.java:40)
at liquibase.executor.jvm.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:72)
at liquibase.snapshot.ResultSetCache$ResultSetExtractor.extract(ResultSetCache.java:314)
at liquibase.snapshot.JdbcDatabaseSnapshot$CachingDatabaseMetaData$GetColumnResultSetCache.extract(JdbcDatabaseSnapshot.java:790)
at liquibase.snapshot.ResultSetCache$ResultSetExtractor.extract(ResultSetCache.java:283)
at liquibase.snapshot.JdbcDatabaseSnapshot$CachingDatabaseMetaData$GetColumnResultSetCache.fastFetchQuery(JdbcDatabaseSnapshot.java:467)
at liquibase.snapshot.ResultSetCache$SingleResultSetExtractor.fastFetch(ResultSetCache.java:338)
at liquibase.snapshot.ResultSetCache.get(ResultSetCache.java:71)

Expected Behaviour

Parsing correctly

Code Example


try (var connection = DriverManager.getConnection(url, info)) {
            connection.setAutoCommit(true);

            var database = new io.goodforgod.liquibase.extension.clickhouse.database.ClickHouseDatabase();
            database.setConnection(new JdbcConnection(connection));
            database.setDefaultCatalogName("default");
            database.setDefaultSchemaName("default");

            Liquibase liquibase = new Liquibase(
                    changeLogFile,
                    new ClassLoaderResourceAccessor(),
                    database
            );
            liquibase.update("");
        }

Environment

  • [ ] Cloud
  • Client version:
  • Language version: java 21
  • OS: Mac

ClickHouse Server

  • ClickHouse Server version: 25.5.1.2782
  • ClickHouse Server non-default settings, if any:
  • CREATE TABLE statements for tables involved:
  • Sample data for all these tables, use clickhouse-obfuscator if necessary

eldarst avatar May 29 '25 10:05 eldarst

@eldarst thank you for reporting! We will look into it.

chernser avatar Jun 02 '25 14:06 chernser

@eldarst I think it is fixed by https://github.com/ClickHouse/clickhouse-java/pull/2480 Would you please take a night build?

chernser avatar Jul 12 '25 06:07 chernser

Hi! The issue still persists in 0.9.1. The debug I've performed shows that despite .transform(...) is added to MetadataResultSet specifically for DATA_TYPE column in DatabaseMetadataImpl.getColumns(...), it doesn't work for Liquibase. The reason is Liquibase reads result set by ResultSet.getObject(...) while MetadataResultSet transformations are considered only when ResultSet.getString(...) is invoked.

Fixed this for now adding clickhouse.jdbc.v2=false (or clickhouse.jdbc.v1=true which is the same) to Liquibase datasource URL in SpringBoot application config file, as follows: spring.liquibase.url=jdbc:ch://localhost:8123?clickhouse.jdbc.v2=false. This helps because the v1 driver converts the name of data type to corresponding integer code right inside the SQL query SELECT clause (ClickhouseDatabaseMetadata.getColumns)

Anyway, I kindly ask the CH team to fix this issue in jdbc-v2! Thank you in advance!

StLt777 avatar Aug 05 '25 20:08 StLt777

Good day, @StLt777 ! Thank you so much for the investigation you have done! Yes we have plans to fix this part in the next release (0.9.2).

chernser avatar Aug 06 '25 00:08 chernser