drill icon indicating copy to clipboard operation
drill copied to clipboard

it cause an exception when query http plugin data with fullName

Open shfshihuafeng opened this issue 1 year ago • 7 comments

Before submitting a bug report, please verify that you are using the most current version of Drill.

Describe the bug i create a http plugin named "http" and then use "use http" command to switch schema. i query http Data with "select * from http.sunrise",It report except

To Reproduce Steps to reproduce the behavior:

1. create plugin with name "http"
2. use http
3. select * from http.sunrise;

Expected behavior

apache drill> select * from http.sunrise;
+----------------------------------------------------------------------------------+--------+
|                                     results                                      | status |
+----------------------------------------------------------------------------------+--------+
| {"sunrise":"5:42:47 AM","sunset":"5:52:15 PM","solar_noon":"11:47:31 AM","day_length":"12:09:28","civil_twilight_begin":"5:21:47 AM","civil_twilight_end":"6:13:15 PM","nautical_twilight_begin":"4:56:00 AM","nautical_twilight_end":"6:39:02 PM","astronomical_twilight_begin":"4:30:08 AM","astronomical_twilight_end":"7:04:54 PM"} | OK     |
+----------------------------------------------------------------------------------+--------+
1 row selected (2.967 seconds)

Error detail, log output or screenshots Error: CONNECTION ERROR: API 'http' does not exist in HTTP storage plugin 'http'

Drill version 1.22.0

Additional context

http plugin config
{
  "type": "http",
  "connections": {
    "sunrise": {
      "url": "https://api.sunrise-sunset.org/json",
      "requireTail": false,
      "method": "GET",
      "params": [
        "lat",
        "lng",
        "date"
      ],
      "authType": "none",
      "inputType": "json",
      "xmlDataLevel": 1,
      "postParameterLocation": "QUERY_STRING",
      "verifySSLCert": true
    }
  },
  "timeout": 5,
  "retryDelay": 1000,
  "proxyType": "direct",
  "authMode": "SHARED_USER",
  "enabled": true
}

shfshihuafeng avatar Nov 27 '23 07:11 shfshihuafeng

we should determines whether the parameter name equals the plug-in name ?

--- a/contrib/storage-http/src/main/java/org/apache/drill/exec/store/http/HttpSchemaFactory.java
+++ b/contrib/storage-http/src/main/java/org/apache/drill/exec/store/http/HttpSchemaFactory.java
@@ -18,8 +18,11 @@
 package org.apache.drill.exec.store.http;

 import java.util.Collections;
+import java.util.Locale;
 import java.util.Map;
 import java.util.Map.Entry;
+import java.util.Set;
+import java.util.stream.Collectors;

 import org.apache.calcite.schema.SchemaPlus;
 import org.apache.calcite.schema.Table;
@@ -85,7 +88,7 @@ public class HttpSchemaFactory extends AbstractSchemaFactory {
       HttpAPIConnectionSchema subSchema = subSchemas.get(name);
       if (subSchema != null) {
         return subSchema;
-      } else if (tables.containsKey(name)) {
+      } else if (tables.containsKey(name) || isRegistryPluginName(name.toLowerCase(Locale.ROOT))) {
         return null;
       } else {
         throw UserException
@@ -95,6 +98,12 @@ public class HttpSchemaFactory extends AbstractSchemaFactory {
       }
     }

+    private boolean isRegistryPluginName(String name) {
+      Set<String> pluginNames = plugin.getRegistry().availablePlugins();
+      Set<String> pluginNamesToLower = pluginNames.stream().map(String::toLowerCase).collect(Collectors.toSet());
+      return pluginNamesToLower.contains(name);
+    }

shfshihuafeng avatar Nov 27 '23 10:11 shfshihuafeng

@shfshihuafeng I don't think this is a bug. You are first running the query USE http which sets the root at http. Then you're running a SELECT ... FROM http.sunrise. So at that point, the plugin is looking for a path: http.http.sunrise which does not exist.

I'd bet that if you either skipped the USE query or ran a SELECT ... FROM sunrise it would work.

cgivre avatar Nov 27 '23 13:11 cgivre

@cgivre You are right, but when use http, the default schema is http, when we use another data source schema, will report error. image It think this is not right.

Javelin2007 avatar Nov 28 '23 00:11 Javelin2007

@cgivre image

shfshihuafeng avatar Nov 28 '23 01:11 shfshihuafeng

@cgivre when i use "use my.test" command to switch schema, It is correct to query data with full schema Name from other data sources 0880c4c80710fc8ae113

shfshihuafeng avatar Nov 28 '23 01:11 shfshihuafeng

@shfshihuafeng So are you saying the behavior with USE is inconsistent? I want to make sure I understand the issue.

cgivre avatar Dec 06 '23 20:12 cgivre

@cgivre There are two problems with this

  1. behavior with USE is inconsistent
  2. when i enter the scehma with 'use',data can not be queried image

shfshihuafeng avatar Dec 11 '23 07:12 shfshihuafeng