datawave icon indicating copy to clipboard operation
datawave copied to clipboard

Regex model expansions

Open ivakegg opened this issue 3 years ago • 0 comments

In issue #1558, we now appropriately push down terms for which the regex normalization cannot be applied to the index. This has caused some issues when we have a regex against a model field which subsequently expands to a mess of database fields and only a few of those cannot be executed using the regex supplied. The feature that we would like is as follows:

  1. Create a query property marker called "ignored" that can wrap terms and will subsequently be ignored for index lookups and evaluation.
  2. When expanding model fields for a regex term, if the regex cannot be normalized for one of the expanded fields which is indexed, then wrap that terms with the ignored query property marker.
  3. If it turns out that all of the field regex terms that the model expanded to cannot be normalized as such, then do not wrap the fields with ignored, and instead wrap the entire expansion with the evaluation only query property marker.

ivakegg avatar Sep 29 '22 14:09 ivakegg