presto icon indicating copy to clipboard operation
presto copied to clipboard

Issue with druid selecting empty columns

Open Yomanz opened this issue 6 months ago • 6 comments

Your Environment

  • Presto version used: 0.288-15f14bb
  • Data source and connector used: Apache Druid, JDBC Connector/SQL Client page on UI
  • Deployment (Cloud or On-prem): On-prem

Expected Behavior

These queries should return the correct data from Apache Druid

Current Behavior

Running the query:

SELECT TABLE_CAT, TABLE_SCHEM, TABLE_NAME, COLUMN_NAME, DATA_TYPE,
  TYPE_NAME, COLUMN_SIZE, BUFFER_LENGTH, DECIMAL_DIGITS, NUM_PREC_RADIX,
  NULLABLE, REMARKS, COLUMN_DEF, SQL_DATA_TYPE, SQL_DATETIME_SUB,
  CHAR_OCTET_LENGTH, ORDINAL_POSITION, IS_NULLABLE,
  SCOPE_CATALOG, SCOPE_SCHEMA, SCOPE_TABLE,
  SOURCE_DATA_TYPE, IS_AUTOINCREMENT, IS_GENERATEDCOLUMN
FROM system.jdbc.columns
ORDER BY TABLE_CAT, TABLE_SCHEM, TABLE_NAME, ORDINAL_POSITION

via the JDBC driver.

Or running: SELECT * FROM table on the SQL CLIENT also gives the same error.

Error:

java.lang.NullPointerException: undefined
	at java.base/java.util.Objects.requireNonNull(Objects.java:221)
	at java.base/java.util.Optional.<init>(Optional.java:107)
	at java.base/java.util.Optional.of(Optional.java:120)
	at com.facebook.presto.druid.DruidMetadata.listTables(DruidMetadata.java:196)
	at com.facebook.presto.druid.DruidMetadata.listTableColumns(DruidMetadata.java:147)
	at com.facebook.presto.metadata.MetadataManager.listTableColumns(MetadataManager.java:562)
	at com.facebook.presto.metadata.MetadataListing.listTableColumns(MetadataListing.java:95)
	at com.facebook.presto.connector.system.jdbc.ColumnJdbcTable.cursor(ColumnJdbcTable.java:126)
	at com.facebook.presto.connector.system.SystemPageSourceProvider$1.cursor(SystemPageSourceProvider.java:130)
	at com.facebook.presto.split.MappedRecordSet.cursor(MappedRecordSet.java:53)
	at com.facebook.presto.spi.RecordPageSource.<init>(RecordPageSource.java:40)
	at com.facebook.presto.connector.system.SystemPageSourceProvider.createPageSource(SystemPageSourceProvider.java:109)
	at com.facebook.presto.spi.connector.ConnectorPageSourceProvider.createPageSource(ConnectorPageSourceProvider.java:55)
	at com.facebook.presto.split.PageSourceManager.createPageSource(PageSourceManager.java:81)
	at com.facebook.presto.operator.TableScanOperator.getOutput(TableScanOperator.java:263)
	at com.facebook.presto.operator.Driver.processInternal(Driver.java:441)
	at com.facebook.presto.operator.Driver.lambda$processFor$10(Driver.java:324)
	at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:750)
	at com.facebook.presto.operator.Driver.processFor(Driver.java:317)
	at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1079)
	at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:165)
	at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:621)
	at com.facebook.presto.$gen.Presto_0_288_15f14bb____20240818_134447_1.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

Possible Solution

I believe it might have something to do with the Apache Druid data having some rows with empty/nulled columns

Steps to Reproduce

  1. Setup Presto
  2. Link Apache Druid
  3. Attempt to run the above queries
  4. See the queries fail with internal errors.

Screenshots (if appropriate)

N/A

Context

Unable to use PrestoDB with tools such as Hex.tech as it cant get the schema.

Yomanz avatar Aug 24 '24 14:08 Yomanz