drill
drill copied to clipboard
Published
20 hours ago •
apache
apache
Trying to use the jdbcldap driver: ERROR com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Exception during pool initialization. com.octetstring.jdbcLdap.jndi.SQLNamingException: Operations Error
trafficstars
Hello
I'm trying to connect Apache Drill to an ldap server through the jdbcldap driver (https://github.com/elbosso/openldap-jdbcldap/)
But Apache drill (1.20) fails with com.octetstring.jdbcLdap.jndi.SQLNamingException: Operations Error
22:13:04.528 [1dbb462f-c8ea-71a8-63fc-3b4a03b7879b:foreman] ERROR com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Exception during pool initialization.
com.octetstring.jdbcLdap.jndi.SQLNamingException: Operations Error
at com.novell.ldap.LDAPResponse.getResultException(LDAPResponse.java:402)
at com.novell.ldap.LDAPResponse.chkResultCode(LDAPResponse.java:365)
at com.novell.ldap.LDAPSearchResults.next(LDAPSearchResults.java:289)
at com.novell.ldap.LDAPConnection.read(LDAPConnection.java:2897)
at com.novell.ldap.LDAPConnection.read(LDAPConnection.java:2864)
at com.novell.ldap.LDAPConnection.fetchSchema(LDAPConnection.java:4187)
at com.octetstring.jdbcLdap.util.TableDef.<init>(TableDef.java:108)
at com.octetstring.jdbcLdap.jndi.JndiLdapConnection.generateTables(JndiLdapConnection.java:819)
at com.octetstring.jdbcLdap.jndi.JndiLdapConnection.<init>(JndiLdapConnection.java:404)
at com.octetstring.jdbcLdap.sql.JdbcLdapDriver.connect(JdbcLdapDriver.java:100)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:81)
at org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.initDataSource(JdbcStoragePlugin.java:161)
at org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>(JdbcStoragePlugin.java:56)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at org.apache.drill.exec.store.ClassicConnectorLocator.create(ClassicConnectorLocator.java:273)
at org.apache.drill.exec.store.ConnectorHandle.newInstance(ConnectorHandle.java:102)
at org.apache.drill.exec.store.PluginHandle.plugin(PluginHandle.java:142)
at org.apache.drill.exec.store.StoragePluginRegistryImpl.getPlugin(StoragePluginRegistryImpl.java:563)
at org.apache.calcite.jdbc.DynamicRootSchema.loadSchemaFactory(DynamicRootSchema.java:107)
at org.apache.calcite.jdbc.DynamicRootSchema.getSchema(DynamicRootSchema.java:87)
at org.apache.calcite.jdbc.DynamicRootSchema.getImplicitSubSchema(DynamicRootSchema.java:73)
at org.apache.calcite.jdbc.CalciteSchema.getSubSchema(CalciteSchema.java:265)
at org.apache.calcite.sql.validate.SqlValidatorUtil.getSchema(SqlValidatorUtil.java:1050)
at org.apache.drill.exec.planner.sql.conversion.DrillCalciteCatalogReader.isValidSchema(DrillCalciteCatalogReader.java:171)
at org.apache.drill.exec.planner.sql.conversion.DrillValidator.validateFrom(DrillValidator.java:66)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3383)
at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1009)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:969)
at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:944)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:651)
at org.apache.drill.exec.planner.sql.conversion.SqlConverter.validate(SqlConverter.java:190)
at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode(DefaultSqlHandler.java:647)
at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert(DefaultSqlHandler.java:195)
at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:169)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:283)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan(DrillSqlWorker.java:163)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan(DrillSqlWorker.java:128)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:93)
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:593)
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:274)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
22:13:05.631 [1dbb462f-c8ea-71a8-63fc-3b4a03b7879b:foreman] ERROR com.zaxxer.hikari.pool.HikariPool - HikariPool-2 - Exception during pool initialization.
com.octetstring.jdbcLdap.jndi.SQLNamingException: Operations Error
I opened also a ticket at https://github.com/elbosso/openldap-jdbcldap/issues/5
Fyi I can connect to ldap (&jdbcldap) using sqlline command line tool after solving some issues with the help of sqlline developers (@snuyanzin and @julianhyde) and @elbosso (https://github.com/julianhyde/sqlline/issues/450)
Thanks in advance Matteo
One guess is that the JDBC driver is trying to fetch LDAP schema information using an invalid DN. Are you using the same URL with Drill as you did with sqlline?
Yes, I use the same connection string as sqlline
I have now simplified my config file removing "table_def" and now I get
java.sql.SQLException: LDAP Does Not Support Transactions
My conf/storage-plugins-override.conf file is now
"storage": {
"adgroup": {
"type" : "jdbc",
"driver" : "com.octetstring.jdbcLdap.sql.JdbcLdapDriver",
"url" : "jdbc:ldap://dmc.group.example.com/DC=group,DC=example,DC=com?SEARCH_SCOPE:=subTreeScope&concat_atts:=true",
# "url" : "jdbc:ldap://dmc.group.example.com/DC=group,DC=exampe,DC=com?SEARCH_SCOPE:=subTreeScope&concat_atts:=true&table_def:=/home/matteo/.config/jdbcldap.conf"
# "username" : "CN=matteo,DC=group,DC=example,DC=com",
"username" : "group\matteo",
"password" : "MySillyPassword",
"enabled" : true
}
}
and I get
matteo@debian:~/apps/apache-drill-1.20.0$ ./bin/drill-embedded
Warning: Nashorn engine is planned to be removed from a future JDK release
Unknown scriptEngine "nashorn". Possible values: [ECMAScript]
Apache Drill 1.20.0
"A query result is never late, nor is it early, it arrives precisely when it means to."
apache drill> select * from adgroup.users limit 1;
09:36:02.438 [1db5601d-16a6-e87c-014c-105f01c0a022:foreman] ERROR com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Error thrown while acquiring connection from data source
java.sql.SQLException: LDAP Does Not Support Transactions
at com.octetstring.jdbcLdap.jndi.JndiLdapConnection.setAutoCommit(JndiLdapConnection.java:531)
at com.zaxxer.hikari.pool.PoolBase.setupConnection(PoolBase.java:413)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:369)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:81)
at org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.initDataSource(JdbcStoragePlugin.java:161)
at org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>(JdbcStoragePlugin.java:56)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at org.apache.drill.exec.store.ClassicConnectorLocator.create(ClassicConnectorLocator.java:273)
at org.apache.drill.exec.store.ConnectorHandle.newInstance(ConnectorHandle.java:102)
at org.apache.drill.exec.store.PluginHandle.plugin(PluginHandle.java:142)
at org.apache.drill.exec.store.StoragePluginRegistryImpl.getPlugin(StoragePluginRegistryImpl.java:563)
at org.apache.calcite.jdbc.DynamicRootSchema.loadSchemaFactory(DynamicRootSchema.java:107)
at org.apache.calcite.jdbc.DynamicRootSchema.getSchema(DynamicRootSchema.java:87)
at org.apache.calcite.jdbc.DynamicRootSchema.getImplicitSubSchema(DynamicRootSchema.java:73)
at org.apache.calcite.jdbc.CalciteSchema.getSubSchema(CalciteSchema.java:265)
at org.apache.calcite.sql.validate.SqlValidatorUtil.getSchema(SqlValidatorUtil.java:1050)
at org.apache.drill.exec.planner.sql.conversion.DrillCalciteCatalogReader.isValidSchema(DrillCalciteCatalogReader.java:171)
at org.apache.drill.exec.planner.sql.conversion.DrillValidator.validateFrom(DrillValidator.java:66)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3383)
at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1009)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:969)
at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:944)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:651)
at org.apache.drill.exec.planner.sql.conversion.SqlConverter.validate(SqlConverter.java:190)
at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode(DefaultSqlHandler.java:647)
at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert(DefaultSqlHandler.java:195)
at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:169)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:283)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan(DrillSqlWorker.java:163)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan(DrillSqlWorker.java:128)
at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:93)
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:593)
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:274)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Try adjusting Hikari settings such as autoCommit and transactionIsolation in a sourceParameters section in your config:
"storage": {
"adgroup": {
"type" : "jdbc",
"driver" : "com.octetstring.jdbcLdap.sql.JdbcLdapDriver",
"url" : "jdbc:ldap://dmc.group.example.com/DC=group,DC=example,DC=com?SEARCH_SCOPE:=subTreeScope&concat_atts:=true",
"sourceParameters" : {
"autoCommit": "foo",
"transactionIsolation": "bar"
},
...
}
}
https://github.com/brettwooldridge/HikariCP