flink-connector-jdbc icon indicating copy to clipboard operation
flink-connector-jdbc copied to clipboard

[FLINK-37288] Add Google Cloud Spanner dialect and catalog

Open laughingman7743 opened this issue 10 months ago • 3 comments

https://issues.apache.org/jira/browse/FLINK-37288

laughingman7743 avatar Feb 09 '25 06:02 laughingman7743

Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)

boring-cyborg[bot] avatar Feb 09 '25 06:02 boring-cyborg[bot]

I have applied code formatting using Spotless and updated the documentation. It is ready for review.

laughingman7743 avatar Feb 09 '25 15:02 laughingman7743

Does this repository not have an owner? Many PRs adding dialects are being left unreviewed. Does this mean that contributions are not accepted?

laughingman7743 avatar May 18 '25 06:05 laughingman7743

CI Failure Investigation

I investigated the CI failure in the compile_and_test (2.1.1, 17) job and found the following:

Error Details

Failed Test: JdbcSourceITCase.testReadWithoutParallelismWithParamsProvider

Root Cause Exception:

java.sql.SQLNonTransientConnectionException: No current connection.
  at org.apache.flink.connector.jdbc.core.datastream.source.reader.JdbcSourceSplitReader.openResultSetForSplitWhenAtLeastOnce
  at org.apache.flink.connector.jdbc.core.datastream.source.reader.JdbcSourceSplitReader.openResultSetForSplitWhenAtMostOnce
  at org.apache.flink.connector.jdbc.core.datastream.source.reader.JdbcSourceSplitReader.openResultSetForSplit

Analysis

  1. Not Spanner-specific: This failure occurs in the flink-connector-jdbc-core module's test, not in Spanner-specific code.

  2. Derby JDBC Connection Issue: The Derby JDBC connection is being closed unexpectedly during ResultSet processing.

  3. Branch Status: This branch is already up-to-date with the latest main branch (commit 669314d), including recent hotfixes:

    • 669314d: Apply the correct version matrix in weekly CI
    • 45cbf78: Change Flink matrix to resolve Flink correctly (#179)
  4. Recent main branch CI: The main branch CI has been passing after these hotfixes.

Local Test Results

I ran the same test locally with both Flink versions:

Flink 2.1.1:

[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.255 s
[INFO] BUILD SUCCESS

Flink 2.0.1:

[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.325 s
[INFO] BUILD SUCCESS

Both tests passed successfully in the local environment.

Conclusion

This appears to be a CI environment-specific issue rather than a code problem:

  • The test passes locally with both Flink 2.1.1 and 2.0.1
  • The issue is in flink-connector-jdbc-core, not Spanner-specific code
  • May be related to timing, resource constraints, or containerization in the CI environment

Recommendation

  • Re-run the CI to see if it's a transient issue
  • This failure should not block the Spanner dialect PR as it's unrelated to the Spanner implementation

laughingman7743 avatar Nov 22 '25 16:11 laughingman7743