solr icon indicating copy to clipboard operation
solr copied to clipboard

SOLR-17195: Add 'minPrefixLength' soft limit

Open gerlowskija opened this issue 8 months ago • 15 comments

https://issues.apache.org/jira/browse/SOLR-17195

Description

Prefix-based queries consume memory in proportion to the number of terms in the index that start with the prefix. Short prefixes tend to match many more indexed terms, and consume more memory as a result, often causing instability issues on the node.

Yet Solr (prior to this PR) offers no way to restrict the prefixes used in queries.

Solution

This PR adds a solrconfig.xml property, minPrefixLength, which operates similarly to the existing maxBooleanClauses. Users who submit a query with a prefix shorter than the minimal acceptable length will receive an error of the form:

Query <snip> does not meet the minimum prefix length [2] (actual=[1]).  Please try with a larger prefix, or adjust minPrefixLength in your solrconfig.xml

Some notes on the implementation:

  • currently only enforced for 'string' and 'text' fields, where this cardinality problem occurs most frequently
  • defaults to '2' in the default configset, prohibiting only single-character prefixes.
  • can be overridden in the default configset with the solr.min.prefixLength sysprop/env-var

Tests

Tests for solrconfig.xml in SolrCoreTest. Tests for the limiting itself in PrefixQueryTest.

Checklist

Please review the following and check all that apply:

  • [x] I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • [x] I have created a Jira issue and added the issue ID to my pull request title.
  • [x] I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
  • [x] I have developed this patch against the main branch.
  • [x] I have run ./gradlew check.
  • [x] I have added tests for my changes.
  • [x] I have added documentation for the Reference Guide

gerlowskija avatar Jun 05 '24 15:06 gerlowskija