tutorials icon indicating copy to clipboard operation
tutorials copied to clipboard
verified publisher icon eugenp

can't find driver class problem in <ETL with Spring Cloud Data Flow>

Open gray19950103 opened this issue 4 years ago • 2 comments

I'm following the ETL with Spring Cloud Data Flow demo on your site. when I get to the

4.2. Spring Cloud Data Flow Server Local

it says in order to load the postgres driver class, need to put jar under a folder and lauch dataflow server with -Dloader.path. I followed all these steps, but when I deploy the stream, the jdbc app would run fail with logs below. any help would be appreciated.

2021-09-24 15:25:52.395  WARN [jdbc-source,,,] 2478 --- [           main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.springframework.cloud.fn.supplier.jdbc.JdbcSupplierConfiguration': Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [org/springframework/boot/autoconfigure/jdbc/DataSourceConfiguration$Hikari.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.zaxxer.hikari.HikariDataSource]: Factory method 'dataSource' threw exception; nested exception is java.lang.IllegalStateException: Cannot load driver class: com.mysql.cj.jdbc.Driver
2021-09-24 15:25:52.399  INFO [jdbc-source,,,] 2478 --- [           main] o.apache.catalina.core.StandardService   : Stopping service [Tomcat]
2021-09-24 15:25:52.470  INFO [jdbc-source,,,] 2478 --- [           main] ConditionEvaluationReportLoggingListener : 

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2021-09-24 15:25:52.518 ERROR [jdbc-source,,,] 2478 --- [           main] o.s.boot.SpringApplication               : Application run failed

org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.springframework.cloud.fn.supplier.jdbc.JdbcSupplierConfiguration': Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [org/springframework/boot/autoconfigure/jdbc/DataSourceConfiguration$Hikari.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.zaxxer.hikari.HikariDataSource]: Factory method 'dataSource' threw exception; nested exception is java.lang.IllegalStateException: Cannot load driver class: com.mysql.cj.jdbc.Driver
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:799) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:228) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1361) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1208) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:556) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:897) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:879) ~[spring-context-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:551) ~[spring-context-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143) ~[spring-boot-2.3.11.RELEASE.jar!/:2.3.11.RELEASE]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:755) [spring-boot-2.3.11.RELEASE.jar!/:2.3.11.RELEASE]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) [spring-boot-2.3.11.RELEASE.jar!/:2.3.11.RELEASE]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:402) [spring-boot-2.3.11.RELEASE.jar!/:2.3.11.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:312) [spring-boot-2.3.11.RELEASE.jar!/:2.3.11.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1247) [spring-boot-2.3.11.RELEASE.jar!/:2.3.11.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1236) [spring-boot-2.3.11.RELEASE.jar!/:2.3.11.RELEASE]
	at org.springframework.cloud.stream.app.jdbc.source.kafka.JdbcSourceKafkaApplication.main(JdbcSourceKafkaApplication.java:29) [classes!/:3.0.2]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_301]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_301]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_301]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_301]
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) [jdbc-source-kafka-3.0.2.jar:3.0.2]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) [jdbc-source-kafka-3.0.2.jar:3.0.2]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) [jdbc-source-kafka-3.0.2.jar:3.0.2]
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88) [jdbc-source-kafka-3.0.2.jar:3.0.2]
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [org/springframework/boot/autoconfigure/jdbc/DataSourceConfiguration$Hikari.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.zaxxer.hikari.HikariDataSource]: Factory method 'dataSource' threw exception; nested exception is java.lang.IllegalStateException: Cannot load driver class: com.mysql.cj.jdbc.Driver
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:657) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:637) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1341) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1181) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:556) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1307) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1227) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:886) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:790) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	... 28 common frames omitted
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.zaxxer.hikari.HikariDataSource]: Factory method 'dataSource' threw exception; nested exception is java.lang.IllegalStateException: Cannot load driver class: com.mysql.cj.jdbc.Driver
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:652) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	... 42 common frames omitted
Caused by: java.lang.IllegalStateException: Cannot load driver class: com.mysql.cj.jdbc.Driver
	at org.springframework.util.Assert.state(Assert.java:97) ~[spring-core-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	at org.springframework.boot.autoconfigure.jdbc.DataSourceProperties.determineDriverClassName(DataSourceProperties.java:224) ~[spring-boot-autoconfigure-2.3.11.RELEASE.jar!/:2.3.11.RELEASE]
	at org.springframework.boot.autoconfigure.jdbc.DataSourceProperties.initializeDataSourceBuilder(DataSourceProperties.java:176) ~[spring-boot-autoconfigure-2.3.11.RELEASE.jar!/:2.3.11.RELEASE]
	at org.springframework.boot.autoconfigure.jdbc.DataSourceConfiguration.createDataSource(DataSourceConfiguration.java:43) ~[spring-boot-autoconfigure-2.3.11.RELEASE.jar!/:2.3.11.RELEASE]
	at org.springframework.boot.autoconfigure.jdbc.DataSourceConfiguration$Hikari.dataSource(DataSourceConfiguration.java:85) ~[spring-boot-autoconfigure-2.3.11.RELEASE.jar!/:2.3.11.RELEASE]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_301]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_301]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_301]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_301]
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.2.15.RELEASE.jar!/:5.2.15.RELEASE]
	... 43 common frames omitted

gray19950103 avatar Sep 24 '21 07:09 gray19950103

Hi @gray19950103 ,

Sorry to hear you're having troubles with our tutorial. I think we need to take a deeper look into this as the article is based on SCDF 1.6.x version whereas the current one is 2.9.x.

kwoyke avatar Nov 16 '21 20:11 kwoyke

Hey @gray19950103 use docker-compose to up scdf. its easy and fast and no need to setup any db. install docker - desktop and docker - compose on you local machine create scdf folder and run below commands - $ curl -sSL https://raw.githubusercontent.com/bitnami/bitnami-docker-spring-cloud-dataflow/master/docker-compose.yml > docker-compose.yml $ docker-compose up -d

after few mins scdf will up in localhost:9393/dashboard

suhitk001 avatar Apr 17 '22 09:04 suhitk001

Hi @gray19950103 I don't know if you still have the issue since it's been a while, but I just checked it out and tried it with spring-cloud-data-flow-server 2.9.2, and the latest JDBC driver for Postgres. It started with no issues. I see in the log that spring is complaining about the MySQL JDBC driver. Please check if you are using Postgres, and make sure you use the appropriate JDBC driver. I put the link here in case you want to download the same cloud-data-flow-server. wget https://repo.spring.io/release/org/springframework/cloud/spring-cloud-dataflow-server/2.9.2/spring-cloud-dataflow-server-2.9.2.jar

ehsansasanian avatar Dec 11 '22 16:12 ehsansasanian

Closed due to lack of activity.

kwoyke avatar Dec 26 '22 12:12 kwoyke

I have jdbc | log plipeline, I get drive class not found maven://org.springframework.cloud.stream.app:jdbc-source-kafka:jar 3.2.1 maven://org.springframework.cloud.stream.app:log-sink-kafka:jar 3.2.1

Error in source app startup Caused by: java.lang.RuntimeException: Failed to load driver class com.mysql.cj.jdbc.Driver in either of HikariConfig class loader or Thread context classloader at com.zaxxer.hikari.HikariConfig.setDriverClassName(HikariConfig.java:491) ~[HikariCP-4.0.3.jar!/:na]

tksarul avatar May 12 '23 10:05 tksarul