druid icon indicating copy to clipboard operation
druid copied to clipboard

Published 20 hours ago • verified publisher icon apache

Bad error message on incorrect context value

Open qoega opened this issue 2 years ago • 2 comments

Query template was incorrect with string value of maxOnDiskStorage context setting. It failed only on the queries that had GROUP BY

debug.json

{"query": "SELECT AdvEngineID, COUNT(*) FROM hits WHERE AdvEngineID <> 0 GROUP BY AdvEngineID ORDER BY COUNT(*) DESC", "context": {"maxOnDiskStorage": "100"} }

How I run query

curl -v -XPOST -H'Content-Type: application/json' http://localhost:8888/druid/v2/sql/ -d @debug.json

Exception message:

{"error":"Unknown exception","errorMessage":"java.lang.String cannot be cast to java.lang.Number","errorClass":"java.lang.ClassCastException","host":null}

Broker stacktrace

2022-07-08T15:51:21,537 WARN [sql[69840d8b-be54-431a-b20f-64caa7636c81]] org.apache.druid.sql.http.SqlResource - Failed to handle query: SqlQuery{query='SELECT AdvEngineID, COUNT(*) FROM hits WHERE AdvEngineID <> 0 GROUP BY AdvEngineID ORDER BY COUNT(*) DESC', resultFormat=object, header=false, typesHeader=false, sqlTypesHeader=false, context={maxOnDiskStorage=100}, parameters=[]}
java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Number
        at org.apache.druid.query.groupby.GroupByQueryConfig.withOverrides(GroupByQueryConfig.java:242) ~[druid-processing-0.23.0.jar:0.23.0]
        at org.apache.druid.query.groupby.strategy.GroupByStrategySelector.strategize(GroupByStrategySelector.java:52) ~[druid-processing-0.23.0.jar:0.23.0]
        at org.apache.druid.query.groupby.GroupByQueryQueryToolChest$6.isCacheable(GroupByQueryQueryToolChest.java:510) ~[druid-processing-0.23.0.jar:0.23.0]
        at org.apache.druid.query.groupby.GroupByQueryQueryToolChest$6.isCacheable(GroupByQueryQueryToolChest.java:502) ~[druid-processing-0.23.0.jar:0.23.0]
        at org.apache.druid.client.CacheUtil.isQueryCacheable(CacheUtil.java:193) ~[druid-server-0.23.0.jar:0.23.0]
        at org.apache.druid.client.CacheUtil.isPopulateResultCache(CacheUtil.java:171) ~[druid-server-0.23.0.jar:0.23.0]
        at org.apache.druid.query.ResultLevelCachingQueryRunner.<init>(ResultLevelCachingQueryRunner.java:74) ~[druid-server-0.23.0.jar:0.23.0]
        at org.apache.druid.server.ClientQuerySegmentWalker.lambda$decorateClusterRunner$1(ClientQuerySegmentWalker.java:440) ~[druid-server-0.23.0.jar:0.23.0]
        at org.apache.druid.query.FluentQueryRunnerBuilder$FluentQueryRunner.map(FluentQueryRunnerBuilder.java:103) ~[druid-processing-0.23.0.jar:0.23.0]
        at org.apache.druid.server.ClientQuerySegmentWalker.decorateClusterRunner(ClientQuerySegmentWalker.java:438) ~[druid-server-0.23.0.jar:0.23.0]
        at org.apache.druid.server.ClientQuerySegmentWalker.getQueryRunnerForIntervals(ClientQuerySegmentWalker.java:205) ~[druid-server-0.23.0.jar:0.23.0]
        at org.apache.druid.query.spec.MultipleIntervalSegmentSpec.lookup(MultipleIntervalSegmentSpec.java:57) ~[druid-processing-0.23.0.jar:0.23.0]
        at org.apache.druid.query.BaseQuery.getRunner(BaseQuery.java:118) ~[druid-processing-0.23.0.jar:0.23.0]
        at org.apache.druid.query.QueryPlus.run(QueryPlus.java:149) ~[druid-processing-0.23.0.jar:0.23.0]
        at org.apache.druid.server.QueryLifecycle.execute(QueryLifecycle.java:283) ~[druid-server-0.23.0.jar:0.23.0]
        at org.apache.druid.server.QueryLifecycle.runSimple(QueryLifecycle.java:160) ~[druid-server-0.23.0.jar:0.23.0]
        at org.apache.druid.sql.calcite.run.NativeQueryMaker.execute(NativeQueryMaker.java:220) ~[druid-sql-0.23.0.jar:0.23.0]
        at org.apache.druid.sql.calcite.run.NativeQueryMaker.runQuery(NativeQueryMaker.java:185) ~[druid-sql-0.23.0.jar:0.23.0]
        at org.apache.druid.sql.calcite.rel.DruidRel.runQuery(DruidRel.java:54) ~[druid-sql-0.23.0.jar:0.23.0]
        at org.apache.druid.sql.calcite.planner.DruidPlanner.lambda$planWithDruidConvention$2(DruidPlanner.java:333) ~[druid-sql-0.23.0.jar:0.23.0]
        at org.apache.druid.sql.calcite.planner.PlannerResult.run(PlannerResult.java:56) ~[druid-sql-0.23.0.jar:0.23.0]
        at org.apache.druid.sql.SqlLifecycle.execute(SqlLifecycle.java:337) ~[druid-sql-0.23.0.jar:0.23.0]
        at org.apache.druid.sql.http.SqlResource.doPost(SqlResource.java:126) ~[druid-sql-0.23.0.jar:0.23.0]
        at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_312]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_312]
        at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) ~[jersey-servlet-1.19.4.jar:1.19.4]
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) ~[jersey-servlet-1.19.4.jar:1.19.4]
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733) ~[jersey-servlet-1.19.4.jar:1.19.4]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) ~[javax.servlet-api-3.1.0.jar:3.1.0]
        at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286) ~[guice-servlet-4.1.0.jar:?]
        at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276) ~[guice-servlet-4.1.0.jar:?]
        at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181) ~[guice-servlet-4.1.0.jar:?]
        at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91) ~[guice-servlet-4.1.0.jar:?]
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85) ~[guice-servlet-4.1.0.jar:?]
        at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:120) ~[guice-servlet-4.1.0.jar:?]
        at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135) ~[guice-servlet-4.1.0.jar:?]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.apache.druid.server.security.PreResponseAuthorizationCheckFilter.doFilter(PreResponseAuthorizationCheckFilter.java:82) ~[druid-server-0.23.0.jar:0.23.0]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.apache.druid.server.initialization.jetty.StandardResponseHeaderFilterHolder$StandardResponseHeaderFilter.doFilter(StandardResponseHeaderFilterHolder.java:161) ~[druid-server-0.23.0.jar:0.23.0]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.apache.druid.server.security.AllowHttpMethodsResourceFilter.doFilter(AllowHttpMethodsResourceFilter.java:78) ~[druid-server-0.23.0.jar:0.23.0]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.apache.druid.server.security.AllowOptionsResourceFilter.doFilter(AllowOptionsResourceFilter.java:75) ~[druid-server-0.23.0.jar:0.23.0]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.apache.druid.server.security.AllowAllAuthenticator$1.doFilter(AllowAllAuthenticator.java:84) ~[druid-server-0.23.0.jar:0.23.0]
        at org.apache.druid.server.security.AuthenticationWrappingFilter.doFilter(AuthenticationWrappingFilter.java:59) ~[druid-server-0.23.0.jar:0.23.0]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.apache.druid.server.security.SecuritySanityCheckFilter.doFilter(SecuritySanityCheckFilter.java:77) ~[druid-server-0.23.0.jar:0.23.0]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) ~[jetty-servlet-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:179) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) ~[jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) [jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) [jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) [jetty-server-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [jetty-io-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882) [jetty-util-9.4.40.v20210413.jar:9.4.40.v20210413]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036) [jetty-util-9.4.40.v20210413.jar:9.4.40.v20210413]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]

qoega avatar Jul 08 '22 15:07 qoega

The maxOnDiskStorage in your json is a type of string, please remove the double quotation marks around the number 100.

Of course from the code side, I think it can be improved to be compatible with type of string.

FrankChen021 avatar Jul 09 '22 02:07 FrankChen021

You can see that I wrote it in first line:

Query template was incorrect with string value of maxOnDiskStorage context setting.

The report is about usability that

  1. No information about what exactly failed to convert.
  2. This conversion is lazy and if query works with context without groupBy it does not mean that same context will work with groupBy.

qoega avatar Jul 11 '22 13:07 qoega

Resolved by #12833

FrankChen021 avatar Sep 02 '22 09:09 FrankChen021