drill
drill copied to clipboard
it cause an exception when query http plugin data with fullName
Before submitting a bug report, please verify that you are using the most current version of Drill.
Describe the bug i create a http plugin named "http" and then use "use http" command to switch schema. i query http Data with "select * from http.sunrise",It report except
To Reproduce Steps to reproduce the behavior:
1. create plugin with name "http"
2. use http
3. select * from http.sunrise;
Expected behavior
apache drill> select * from http.sunrise;
+----------------------------------------------------------------------------------+--------+
| results | status |
+----------------------------------------------------------------------------------+--------+
| {"sunrise":"5:42:47 AM","sunset":"5:52:15 PM","solar_noon":"11:47:31 AM","day_length":"12:09:28","civil_twilight_begin":"5:21:47 AM","civil_twilight_end":"6:13:15 PM","nautical_twilight_begin":"4:56:00 AM","nautical_twilight_end":"6:39:02 PM","astronomical_twilight_begin":"4:30:08 AM","astronomical_twilight_end":"7:04:54 PM"} | OK |
+----------------------------------------------------------------------------------+--------+
1 row selected (2.967 seconds)
Error detail, log output or screenshots
Error: CONNECTION ERROR: API 'http' does not exist in HTTP storage plugin 'http'
Drill version 1.22.0
Additional context
http plugin config
{
"type": "http",
"connections": {
"sunrise": {
"url": "https://api.sunrise-sunset.org/json",
"requireTail": false,
"method": "GET",
"params": [
"lat",
"lng",
"date"
],
"authType": "none",
"inputType": "json",
"xmlDataLevel": 1,
"postParameterLocation": "QUERY_STRING",
"verifySSLCert": true
}
},
"timeout": 5,
"retryDelay": 1000,
"proxyType": "direct",
"authMode": "SHARED_USER",
"enabled": true
}
we should determines whether the parameter name equals the plug-in name ?
--- a/contrib/storage-http/src/main/java/org/apache/drill/exec/store/http/HttpSchemaFactory.java
+++ b/contrib/storage-http/src/main/java/org/apache/drill/exec/store/http/HttpSchemaFactory.java
@@ -18,8 +18,11 @@
package org.apache.drill.exec.store.http;
import java.util.Collections;
+import java.util.Locale;
import java.util.Map;
import java.util.Map.Entry;
+import java.util.Set;
+import java.util.stream.Collectors;
import org.apache.calcite.schema.SchemaPlus;
import org.apache.calcite.schema.Table;
@@ -85,7 +88,7 @@ public class HttpSchemaFactory extends AbstractSchemaFactory {
HttpAPIConnectionSchema subSchema = subSchemas.get(name);
if (subSchema != null) {
return subSchema;
- } else if (tables.containsKey(name)) {
+ } else if (tables.containsKey(name) || isRegistryPluginName(name.toLowerCase(Locale.ROOT))) {
return null;
} else {
throw UserException
@@ -95,6 +98,12 @@ public class HttpSchemaFactory extends AbstractSchemaFactory {
}
}
+ private boolean isRegistryPluginName(String name) {
+ Set<String> pluginNames = plugin.getRegistry().availablePlugins();
+ Set<String> pluginNamesToLower = pluginNames.stream().map(String::toLowerCase).collect(Collectors.toSet());
+ return pluginNamesToLower.contains(name);
+ }
@shfshihuafeng I don't think this is a bug. You are first running the query USE http
which sets the root at http
. Then you're running a SELECT ... FROM http.sunrise
. So at that point, the plugin is looking for a path: http.http.sunrise
which does not exist.
I'd bet that if you either skipped the USE
query or ran a SELECT ... FROM sunrise
it would work.
@cgivre You are right, but when use http, the default schema is http, when we use another data source schema, will report error.
It think this is not right.
@cgivre
@cgivre
when i use "use my.test" command to switch schema, It is correct to query data with full schema Name from other data sources
@shfshihuafeng
So are you saying the behavior with USE
is inconsistent? I want to make sure I understand the issue.
@cgivre There are two problems with this
- behavior with USE is inconsistent
- when i enter the scehma with 'use',data can not be queried