quickwit icon indicating copy to clipboard operation
quickwit copied to clipboard

Range Query on str field not supported (contrary to what docs imply)

Open RaphaelMarinier opened this issue 2 months ago • 0 comments

Describe the bug

Range queries on fast str fields are not supported. However, the documentation seems to imply so:

https://quickwit.io/docs/reference/query-language#range-fieldlow_bound-to-high_bound

Order For text fields, the ranges are defined by lexicographic order on uft-8 encoded byte arrays. It means for a text field, 100 is between 1 and 2.

https://quickwit.io/docs/get-started/query-language-intro includes the following range query example:

last_name:[banner TO miller]

Steps to reproduce (if applicable) Steps to reproduce the behavior:

  1. Have an index with a trace_id text fast field:
doc_mapping:
  mode: lenient
  field_mappings:
    - name: timestamp
      type: datetime
      precision: milliseconds
      fast: true
      input_formats:
        - rfc3339
      output_format: unix_timestamp_secs
    - name: trace_id
      type: text
      tokenizer: raw
      fast: true
  timestamp_field: timestamp

(Having an explicit schema instead of a dynamic one is important - another bug report is coming)

  1. Query that field:
quickwit tool local-search --index generated-logs-tiny-fast  --config quickwit.yaml --query 'trace_id:[abc TO def]'

❯ Searching directly on the index storage (without calling REST API)... 2024-06-07T09:55:19.575Z ERROR quickwit: command failed error=invalid query: range query on str field (trace_id) forbidden ✘ command failed: invalid query: range query on str field (trace_id) forbidden

Note: using quickwit index search does not change the outcome:

quickwit index search --index generated-logs-tiny-fast  --query 'trace_id:[abc TO def]'

2024-06-07T10:04:24.052Z ERROR quickwit: command failed error=API error: (code=400 Bad Request, message=invalid query: range query on str field (trace_id) forbidden) ✘ command failed: API error: (code=400 Bad Request, message=invalid query: range query on str field (trace_id) forbidden)

Caused by: (code=400 Bad Request, message=invalid query: range query on str field (trace_id) forbidden)

Expected behavior Either update the documentation to mention that range queries on fast text fields are not supported, or support them. The error is raised in range_query.rs.

Configuration:

  1. quickwit --version Quickwit 0.8.0 (x86_64-unknown-linux-gnu 2024-06-06T20:01:05Z 1fd7d0b)

  2. The index_config.yaml index-config.quickwit.yaml.txt

RaphaelMarinier avatar Jun 07 '24 10:06 RaphaelMarinier