vespa
vespa copied to clipboard
Published
20 hours ago •
vespa-engine
vespa-engine
Validate attribute references in grouping in the container
Referencing non-existing attributes in grouping expressions lead to a warning in the log:
searchnode proton.groupingmanager error Could not locate attribute for grouping number 0 : Failed locating attribute vector 'fieldname'. Ignoring grouping 'search::aggregation::Grouping {\n id: 0\n valid: true\n all: false\n topN: -1\n firstLevel: 0\n lastLevel: 1\n levels: std::vector {\n [0]: search::aggregation::GroupingLevel {\n
But no error in the result.
Please attach exact details on how to reproduce this. This is what I get when referencing a non-existing attribute in grouping:
[08:06:54.630] RESULT: <?xml version="1.0" encoding="utf-8" ?>
[08:06:54.630] <result total-hit-count="0">
[08:06:54.630] <error code="5">Unspecified error</error>
[08:06:54.630] <meta type="context">
[08:06:54.630] <p>No query profile is used</p>
[08:06:54.630] <p>Invoking chain 'vespa' [com.yahoo.search.querytransform.WeakAndReplacementSearcher@vespa -> com.yahoo.prelude.statistics.StatisticsSearcher@native -> ... -> federation@native]</p>
[08:06:54.630] <p>
[08:06:54.630] <p>Federating to [search]</p>
[08:06:54.630] <p>Query parsed to: select * from sources * where sddocname contains "test" timeout 10000 | all(all(group(a) each(output(count()))) all(group(e) each(output(count()))));</p>
[08:06:54.630] </p>
[08:06:54.630] </meta>
[08:06:54.630] <errordetails>
[08:06:54.630] <error error="Unspecified error" code="5">Failed: Grouping request references attribute 'e' which is not available in cluster 'search'.
[08:06:54.630] <cause>
[08:06:54.630] com.yahoo.search.grouping.UnavailableAttributeException: Grouping request references attribute 'e' which is not available in cluster 'search'.
[08:06:54.630] at com.yahoo.search.grouping.GroupingValidator.verifyHasAttribute(GroupingValidator.java:75)
[08:06:54.630] at com.yahoo.search.grouping.GroupingValidator$MyVisitor.visitExpression(GroupingValidator.java:108)
[08:06:54.630] at com.yahoo.search.grouping.request.GroupingExpression.visit(GroupingExpression.java:66)
[08:06:54.630] at com.yahoo.search.grouping.request.GroupingOperation.visitExpressions(GroupingOperation.java:497)
[08:06:54.630] at com.yahoo.search.grouping.request.GroupingOperation.visitExpressions(GroupingOperation.java:500)
[08:06:54.630] at com.yahoo.search.grouping.GroupingValidator.search(GroupingValidator.java:68)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.search.searchers.QueryValidator.search(QueryValidator.java:33)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.search.querytransform.SortingDegrader.search(SortingDegrader.java:52)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.search.querytransform.RangeQueryOptimizer.search(RangeQueryOptimizer.java:43)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.prelude.querytransform.LiteralBoostSearcher.search(LiteralBoostSearcher.java:38)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.prelude.querytransform.CJKSearcher.search(CJKSearcher.java:43)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.search.querytransform.DefaultPositionSearcher.search(DefaultPositionSearcher.java:43)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.search.querytransform.NGramSearcher.search(NGramSearcher.java:50)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.search.federation.FederationSearcher.search(FederationSearcher.java:233)
[08:06:54.630] at com.yahoo.search.federation.FederationSearcher.search(FederationSearcher.java:207)
[08:06:54.630] at com.yahoo.search.federation.FederationSearcher.search(FederationSearcher.java:202)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.prelude.searcher.BlendingSearcher.search(BlendingSearcher.java:59)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.search.grouping.GroupingQueryParser.search(GroupingQueryParser.java:56)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.prelude.semantics.SemanticSearcher.search(SemanticSearcher.java:88)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.prelude.searcher.PosSearcher.search(PosSearcher.java:63)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.prelude.searcher.JuniperSearcher.search(JuniperSearcher.java:69)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.search.yql.FieldFilter.search(FieldFilter.java:34)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.search.yql.MinimalQueryInserter.search(MinimalQueryInserter.java:78)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.prelude.searcher.FieldCollapsingSearcher.search(FieldCollapsingSearcher.java:101)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.prelude.querytransform.PhrasingSearcher.search(PhrasingSearcher.java:60)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.prelude.statistics.StatisticsSearcher.search(StatisticsSearcher.java:255)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.search.querytransform.WeakAndReplacementSearcher.search(WeakAndReplacementSearcher.java:22)
[08:06:54.630] at com.yahoo.search.Searcher.process(Searcher.java:134)
[08:06:54.630] at com.yahoo.processing.execution.Execution.process(Execution.java:112)
[08:06:54.630] at com.yahoo.search.searchchain.Execution.search(Execution.java:493)
[08:06:54.630] at com.yahoo.search.handler.SearchHandler.searchAndFill(SearchHandler.java:448)
[08:06:54.630] at com.yahoo.search.handler.SearchHandler.search(SearchHandler.java:498)
[08:06:54.630] at com.yahoo.search.handler.SearchHandler.handleBody(SearchHandler.java:367)
[08:06:54.630] at com.yahoo.search.handler.SearchHandler.handle(SearchHandler.java:281)
[08:06:54.630] at com.yahoo.container.jdisc.ThreadedHttpRequestHandler.handle(ThreadedHttpRequestHandler.java:71)
[08:06:54.631] at com.yahoo.container.jdisc.ThreadedHttpRequestHandler.handleRequest(ThreadedHttpRequestHandler.java:82)
[08:06:54.631] at com.yahoo.container.jdisc.ThreadedRequestHandler$RequestTask.processRequest(ThreadedRequestHandler.java:191)
[08:06:54.631] at com.yahoo.container.jdisc.ThreadedRequestHandler$RequestTask.run(ThreadedRequestHandler.java:185)
[08:06:54.631] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[08:06:54.631] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[08:06:54.631] at java.base/java.lang.Thread.run(Thread.java:834)
[08:06:54.631]
[08:06:54.631] </cause>
[08:06:54.631] </error>
[08:06:54.631] </errordetails>
[08:06:54.631] </result>
You need to have multiple document types where at least one has the attribute, and at least one does not. Then you should trigger this case when you query both.
We currently get this error in the JSON result (via issue reporting):
{
"message":
"Could not locate attribute for grouping number 1 : Failed locating attribute vector 'e'. Ignoring this grouping.",
"summary" : "Error in search reply.",
"source" : "search",
"code" : 8
}
Full JSON:
{"root":{"id":"toplevel","relevance":1.0,"fields":{"totalCount":3},"coverage":{"coverage":100,"documents":6,"full":true,"nodes":2,"results":2,"resultsFull":2},"errors":[{"code":8,"summary":"Error in search reply.","source":"search","message":"Could not locate attribute for grouping number 1 : Failed locating attribute vector 'e'. Ignoring this grouping."}],"children":[{"id":"id:test1:test1::3","relevance":1.5,"source":"search","fields":{"sddocname":"test1","documentid":"id:test1:test1::3","a":"a2","b":"b2"}},{"id":"group:root:0","relevance":1.0,"continuation":{"this":""},"children":[{"id":"grouplist:a","relevance":1.0,"label":"a","children":[{"id":"group:string:a2","relevance":1.5,"value":"a2","fields":{"count()":3}}]},{"id":"grouplist:b","relevance":1.0,"label":"b","children":[{"id":"group:string:b2","relevance":0.75,"value":"b2","fields":{"count()":1,"sum(e)":0.75}},{"id":"group:string:b1","relevance":0.5,"value":"b1","fields":{"count()":1,"sum(e)":0.5}}]}]},{"id":"id:test2:test2::3","relevance":0.75,"source":"search","fields":{"sddocname":"test2","documentid":"id:test2:test2::3","a":"a2","b":"b2"}},{"id":"id:test2:test2::2","relevance":0.5,"source":"search","fields":{"sddocname":"test2","documentid":"id:test2:test2::2","a":"a2","b":"b1"}}]}}
@bratseth is this enough or should we validate (and fail query entirely) in QRS?