group_by pagination stops returning results beyond some limit in Typesense
Bug Description
When using group_by with a high number of unique groups, Typesense only returns results up to the max_query_groups limit. Requests for higher page numbers return empty grouped_hits, even though there are more unique groups available in the collection.
Reproduction Steps
curl -X GET "http://TS_URL/collections/products/documents/search"
-H "X-TYPESENSE-API-KEY:API_KEY"
-G --data-urlencode 'q=*'
--data-urlencode 'query_by=product_id'
--data-urlencode 'filter_by=(brand_name:="Nike")'
--data-urlencode 'per_page=10'
--data-urlencode 'page=12'
--data-urlencode 'group_by=product_id'
--data-urlencode 'group_limit=1'
--data-urlencode 'sort_by=product_id:asc'
Expected vs Actual
It should give result for page 12, give 10 records
Actual behavior
{"facet_counts":[],"found":13061,"found_docs":51947,"grouped_hits":[],"out_of":435977,"page":12,"request_params":{"collection_name":"products","first_q":"","per_page":10,"q":""},"search_cutoff":false,"search_time_ms":96}
It gives grouped_hits as blank, for page 11 it works.
Environment
-
Typesense version: e.g.
v29.0
Schema / Configuration
Include any relevant schema definitions, JSON payloads, or config files:
schema = {
"name": "products",
"enable_nested_fields": True,
"fields": [
{"name": "id", "type": "string", "facet": True, "sort": True},
{"name": "product_id", "type": "string", "sort": True},
{
"name": "images",
"type": "object[]",
"fields": [
{"name": "product_image_id", "type": "string", "sort": True},
{"name": "is_primary", "type": "bool", "sort": True},
{"name": "large_url", "type": "string"}
]
},
{
"name": "store_details",
"type": "object",
"fields": [
{"name": "store_id", "type": "string", "sort": True},
]
}
],
"default_sorting_field": "price_amount",
"num_shards": 4,
}
Additional Context
For other product brands some works till 150 page for 200 page does not work. Total pages in this case was 1800+
@DineshC1994 Can you try again with the latest RC version, 30.0.rc25?
@happy-san Thanks, after upgrading to v30rc26 it is resolved.
Can i know when v30 will be officially release so can plan our production accordingly as currently v29 is showing as latest release.
Currently v29 is showing as latest release.
Where?
@tharropoulos , I have attached details below.
Link: https://typesense.org/downloads#:~:text=Copy%20Code-,docker%20pull%20typesense/typesense%3A29.0,-Linux Github Release image attached