flink-cdc icon indicating copy to clipboard operation
flink-cdc copied to clipboard

Overlapping definitions with PostgreSQL JDBC driver

Open sebasmagri opened this issue 4 years ago • 1 comments

When using both postgresql and flink-connector-postgres-cdc in a project as follows:

		<dependency>
			<groupId>org.postgresql</groupId>
			<artifactId>postgresql</artifactId>
			<version>${postgresql.version}</version>
		</dependency>
		<dependency>
			<groupId>org.apache.flink</groupId>
			<artifactId>flink-connector-jdbc_${scala.binary.version}</artifactId>
			<version>${flink.version}</version>
		</dependency>
		<dependency>
			<groupId>com.alibaba.ververica</groupId>
			<artifactId>flink-connector-postgres-cdc</artifactId>
			<version>1.1.0</version>
		</dependency>

I'm getting this output during build/packaging:

[WARNING] flink-connector-postgres-cdc-1.1.0.jar, postgresql-42.2.18.jre7.jar define 403 overlapping classes:
[WARNING]   - org.postgresql.core.BaseStatement
[WARNING]   - org.postgresql.translation.messages_de
[WARNING]   - org.postgresql.shaded.com.ongres.scram.common.stringprep.StringPreparations$1
[WARNING]   - org.postgresql.replication.LogSequenceNumber
[WARNING]   - org.postgresql.jdbc.PgConnection
[WARNING]   - org.postgresql.replication.fluent.ChainedCommonStreamBuilder
[WARNING]   - org.postgresql.util.GT
[WARNING]   - org.postgresql.xa.PGXADataSourceFactory
[WARNING]   - org.postgresql.osgi.PGDataSourceFactory
[WARNING]   - org.postgresql.core.v3.Portal
[WARNING]   - 393 more...
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/

This in turn causes the final project to not be able to find either of the table connectors depending on their position in the dependencies, and getting an exception like this when trying to use it:

Caused by: org.apache.flink.table.api.ValidationException: Could not find any factory for identifier 'postgres-cdc' that implements 'org.apache.flink.table.factories.DynamicTableFactory' in the classpath.

Available factory identifiers are:

blackhole
datagen
filesystem
jdbc
print

sebasmagri avatar Jan 25 '21 16:01 sebasmagri

#1203 I commented below with a gradle-version solution. In the case of maven, try adding

1161658392894_ pic

to your pom.xml

hbk671104 avatar Jul 21 '22 08:07 hbk671104

Closing this issue because it was created before version 2.3.0 (2022-11-10). Please try the latest version of Flink CDC to see if the issue has been resolved. If the issue is still valid, kindly report it on Apache Jira under project Flink with component tag Flink CDC. Thank you!

PatrickRen avatar Feb 28 '24 15:02 PatrickRen