sql
sql copied to clipboard
[BUG] NPE when create Spark data source
What is the bug?
NPE when create Spark data source
How can one reproduce the bug?
curl --request POST \
--url localhost:9200/_plugins/_query/_datasources \
--header 'content-type: application/x-ndjson' \
--data '{"name": "mys5","description": "","connector": "SPARK","allowedRoles": [],"properties": {"glue.auth.type": "iam_role","glue.auth.role_arn": "arn:aws:iam::123456:role/flint-opensearch-role","glue.indexstore.opensearch.uri": "http://localhost:9200","glue.indexstore.opensearch.auth": "noauth"}}'
{
"status": 400,
"error": {
"type": "NullPointerException",
"reason": "Invalid Request",
"details": "Cannot invoke \"String.equals(Object)\" because the return value of \"java.util.Map.get(Object)\" is null"
}
}%
## opensearch.log
Error happened during request handling
java.lang.NullPointerException: Cannot invoke "String.equals(Object)" because the return value of "java.util.Map.get(Object)" is null
at org.opensearch.sql.spark.storage.SparkStorageFactory.getStorageEngine(SparkStorageFactory.java:78) ~[spark-3.0.0.0-SNAPSHOT.jar:?]
at org.opensearch.sql.spark.storage.SparkStorageFactory.createDataSource(SparkStorageFactory.java:67) ~[spark-3.0.0.0-SNAPSHOT.jar:?]
at org.opensearch.sql.datasources.service.DataSourceLoaderCacheImpl.getOrLoadDataSource(DataSourceLoaderCacheImpl.java:43) ~[datasources-3.0.0.0-SNAPSHOT.jar:?]
at org.opensearch.sql.datasources.service.DataSourceServiceImpl.createDataSource(DataSourceServiceImpl.java:85) ~[datasources-3.0.0.0-SNAPSHOT.jar:?]
at org.opensearch.sql.datasources.transport.TransportCreateDataSourceAction.doExecute(TransportCreateDataSourceAction.java:73) [datasources-3.0.0.0-SNAPSHOT.jar:?]
at org.opensearch.sql.datasources.transport.TransportCreateDataSourceAction.doExecute(TransportCreateDataSourceAction.java:27) [datasources-3.0.0.0-SNAPSHOT.jar:?]
at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:218) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.opensearch.action.support.TransportAction.execute(TransportAction.java:188) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.opensearch.action.support.TransportAction.execute(TransportAction.java:107) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:476) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at org.opensearch.sql.datasources.rest.RestDataSourceQueryAction.lambda$executePostRequest$1(RestDataSourceQueryAction.java:145) [datasources-3.0.0.0-SNAPSHOT.jar:?]
at org.opensearch.sql.datasources.utils.Scheduler.lambda$withCurrentContext$0(Scheduler.java:30) [datasources-3.0.0.0-SNAPSHOT.jar:?]
at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:854) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
What is the expected behavior? A clear and concise description of what you expected to happen.
If it's due to config missing, we should handle this gracefully.
What is your host/environment?
- OS: [e.g. iOS]
- Version [e.g. 22]
- Plugins
OS main sql main
curl -XPOST localhost:9200/_plugins/_query/_datasources \
--header 'content-type: application/x-ndjson' \
--data '{"name": "mys5","description": "","connector": "SPARK","allowedRoles": [],"properties": {"glue.auth.type": "iam_role","glue.auth.role_arn": "arn:aws:iam::123456:role/flint-opensearch-role","glue.indexstore.opensearch.uri": "http://localhost:9200","glue.indexstore.opensearch.auth": "noauth"}}' -u admin:admin --insecure
{
"status": 400,
"error": {
"type": "NullPointerException",
"reason": "Invalid Request",
"details": "Cannot invoke \"String.equals(Object)\" because the return value of \"java.util.Map.get(Object)\" is null"
}
}