presto
presto copied to clipboard
Issue with druid selecting empty columns
Your Environment
- Presto version used: 0.288-15f14bb
- Data source and connector used: Apache Druid, JDBC Connector/SQL Client page on UI
- Deployment (Cloud or On-prem): On-prem
Expected Behavior
These queries should return the correct data from Apache Druid
Current Behavior
Running the query:
SELECT TABLE_CAT, TABLE_SCHEM, TABLE_NAME, COLUMN_NAME, DATA_TYPE,
TYPE_NAME, COLUMN_SIZE, BUFFER_LENGTH, DECIMAL_DIGITS, NUM_PREC_RADIX,
NULLABLE, REMARKS, COLUMN_DEF, SQL_DATA_TYPE, SQL_DATETIME_SUB,
CHAR_OCTET_LENGTH, ORDINAL_POSITION, IS_NULLABLE,
SCOPE_CATALOG, SCOPE_SCHEMA, SCOPE_TABLE,
SOURCE_DATA_TYPE, IS_AUTOINCREMENT, IS_GENERATEDCOLUMN
FROM system.jdbc.columns
ORDER BY TABLE_CAT, TABLE_SCHEM, TABLE_NAME, ORDINAL_POSITION
via the JDBC driver.
Or running: SELECT * FROM table
on the SQL CLIENT also gives the same error.
Error:
java.lang.NullPointerException: undefined
at java.base/java.util.Objects.requireNonNull(Objects.java:221)
at java.base/java.util.Optional.<init>(Optional.java:107)
at java.base/java.util.Optional.of(Optional.java:120)
at com.facebook.presto.druid.DruidMetadata.listTables(DruidMetadata.java:196)
at com.facebook.presto.druid.DruidMetadata.listTableColumns(DruidMetadata.java:147)
at com.facebook.presto.metadata.MetadataManager.listTableColumns(MetadataManager.java:562)
at com.facebook.presto.metadata.MetadataListing.listTableColumns(MetadataListing.java:95)
at com.facebook.presto.connector.system.jdbc.ColumnJdbcTable.cursor(ColumnJdbcTable.java:126)
at com.facebook.presto.connector.system.SystemPageSourceProvider$1.cursor(SystemPageSourceProvider.java:130)
at com.facebook.presto.split.MappedRecordSet.cursor(MappedRecordSet.java:53)
at com.facebook.presto.spi.RecordPageSource.<init>(RecordPageSource.java:40)
at com.facebook.presto.connector.system.SystemPageSourceProvider.createPageSource(SystemPageSourceProvider.java:109)
at com.facebook.presto.spi.connector.ConnectorPageSourceProvider.createPageSource(ConnectorPageSourceProvider.java:55)
at com.facebook.presto.split.PageSourceManager.createPageSource(PageSourceManager.java:81)
at com.facebook.presto.operator.TableScanOperator.getOutput(TableScanOperator.java:263)
at com.facebook.presto.operator.Driver.processInternal(Driver.java:441)
at com.facebook.presto.operator.Driver.lambda$processFor$10(Driver.java:324)
at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:750)
at com.facebook.presto.operator.Driver.processFor(Driver.java:317)
at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1079)
at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:165)
at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:621)
at com.facebook.presto.$gen.Presto_0_288_15f14bb____20240818_134447_1.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Possible Solution
I believe it might have something to do with the Apache Druid data having some rows with empty/nulled columns
Steps to Reproduce
- Setup Presto
- Link Apache Druid
- Attempt to run the above queries
- See the queries fail with internal errors.
Screenshots (if appropriate)
N/A
Context
Unable to use PrestoDB with tools such as Hex.tech as it cant get the schema.