datawave icon indicating copy to clipboard operation
datawave copied to clipboard

QueryMetrics writing should handle query parse error better

Open billoley opened this issue 3 years ago • 0 comments

Queries that contain lucene functions that are not configured in the LuceneToJexlQueryParser will cause an NPE in ContentQueryMetricsIngestHelper which is not caught locally enough to allow the query metric to continue being written. QueryMetrics with Lifecycle NONE were observed. Any query that fails to parse would cause the same issue.

This happened because the code was using a default LuceneToJexlQueryParser which contained a default set of allowedFunctions. If the QueryLogicFactory.xml defined allowedFunctions has entries that are not in the default list (see #1547), then this issue will occur.

  1. Inject the LuceneToJexlQueryParser @SpringBean into the ShardTableQueryMetricHandler
  2. Better define when POSITIVE_SELECTORS and NEGATIVE_SELECTORS should be extracted (was using numUpdates)
  3. Ensure that numUpdates is being properly updates
  4. Catch Exception when parsing the query to extract POSITIVE_SELECTORS and NEGATIVE_SELECTORS
  5. Move parsing from ContentQueryMetricsIngestHelper to BaseQueryMetricHandler, allowing the cached QueryMetric to be updated

Separately -- another issue should look at the LuceneToJexlQueryParser used for selector eextraction for audit purposes (DatawaveSelectorExtractor also uses the default allowedFunctions)

billoley avatar May 06 '22 19:05 billoley