[BUG] not all records matching the dashboards current search criteria are being displayed in Data Table

[pslusarz]

trafficstars

Describe the bug

One of my data tables, which has aggregations across several fields is not displaying all the records that it should.

To Reproduce Steps to reproduce the behavior:

1. Go to dashboard
2. In category-picker data table, select "cargo capacity", filter for value
3. Note all counts on the other data tables total to 20, except review-detail one (bottom). There should be 10 rows there, with Category count = 2, each.

Expected behavior There should be 10 rows there, with Category count = 2, each.

OpenSearch Version

Dashboards Version 2.0.2 (via AWS Open Search managed)

Plugins

n/a

Screenshots Dashboard with the issue demonstrated:

Index definition (note, this was automatically generated after the data was loaded)

Configuration of the data table causing the error (several screenshots, since I had to expand each bucket configuration):

If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

• OS: OSX Sonoma 14.1.2
• Chrome Version 122.0.6261.94 (Official Build) (x86_64)

Additional context

Request from the Data Table: { "aggs": { "4": { "terms": { "field": "review_id.keyword", "order": { "_count": "desc" }, "size": 99999 }, "aggs": { "2": { "terms": { "field": "title.keyword", "order": { "_count": "desc" }, "size": 99999 }, "aggs": { "3": { "terms": { "field": "comments.keyword", "order": { "_count": "desc" }, "size": 99999 }, "aggs": { "6": { "terms": { "field": "make.keyword", "order": { "_count": "desc" }, "size": 99999 }, "aggs": { "7": { "terms": { "field": "model.keyword", "order": { "_count": "desc" }, "size": 99999 }, "aggs": { "8": { "terms": { "field": "year.keyword", "order": { "_count": "desc" }, "size": 99999 } } } } } } } } } } } } }, "size": 0, "stored_fields": [ "*" ], "script_fields": {}, "docvalue_fields": [ { "field": "@timestamp", "format": "date_time" } ], "_source": { "excludes": [] }, "query": { "bool": { "must": [], "filter": [ { "match_all": {} }, { "match_phrase": { "category.keyword": "cargo capacity" } }, { "range": { "@timestamp": { "gte": "2009-03-07T21:01:09.149Z", "lte": "2024-03-07T21:01:09.149Z", "format": "strict_date_optional_time" } } } ], "should": [], "must_not": [] } } }

Response: { "took": 9, "timed_out": false, "_shards": { "total": 0, "successful": 0, "skipped": 0, "failed": 0 }, "hits": { "total": 20, "max_score": null, "hits": [] }, "aggregations": { "4": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "2": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "3": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [] }, "key": "its ok, not loving it. ", "doc_count": 2 } ] }, "key": "1692265", "doc_count": 2 }, { "2": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "3": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [] }, "key": "LOVE IT", "doc_count": 2 } ] }, "key": "2507723", "doc_count": 2 }, { "2": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "3": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "6": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "7": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "8": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": "2021", "doc_count": 2 } ] }, "key": "ODYSSEY", "doc_count": 2 } ] }, "key": "HONDA", "doc_count": 2 } ] }, "key": "Probably great for bigger families, or someone who hauls a lot of stuff. Guzzles a LOT of gas, haven’t had it a year & it’s already noisy at idle.", "doc_count": 2 } ] }, "key": "Disappointed ", "doc_count": 2 } ] }, "key": "2694723", "doc_count": 2 }, { "2": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "3": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "6": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "7": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "8": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": "2019", "doc_count": 2 } ] }, "key": "ODYSSEY", "doc_count": 2 } ] }, "key": "HONDA", "doc_count": 2 } ] }, "key": "Consider the seating options and the ability to do jobs that you use to use a small truck for.", "doc_count": 2 } ] }, "key": "Great vehicle for doing every job.", "doc_count": 2 } ] }, "key": "448153", "doc_count": 2 }, { "2": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "3": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "6": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "7": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "8": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": "2020", "doc_count": 2 } ] }, "key": "ODYSSEY", "doc_count": 2 } ] }, "key": "HONDA", "doc_count": 2 } ] }, "key": "Well-built vehicle that is spacious and easily seats 8 passengers, and provides lots of cargo space.", "doc_count": 2 } ] }, "key": "Great Minivan that offers really comfortable and smooth drive. ", "doc_count": 2 } ] }, "key": "5116113", "doc_count": 2 }, { "2": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "3": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [] }, "key": "A good mini van", "doc_count": 2 } ] }, "key": "5327325", "doc_count": 2 }, { "2": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "3": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "6": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "7": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "8": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": "2022", "doc_count": 2 } ] }, "key": "ODYSSEY", "doc_count": 2 } ] }, "key": "HONDA", "doc_count": 2 } ] }, "key": "Great for long trips with the family. Need more cargo space", "doc_count": 2 } ] }, "key": "Comfortable ride for the family", "doc_count": 2 } ] }, "key": "5356973", "doc_count": 2 }, { "2": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "3": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "6": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "7": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "8": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": "2019", "doc_count": 2 } ] }, "key": "ODYSSEY", "doc_count": 2 } ] }, "key": "HONDA", "doc_count": 2 } ] }, "key": "Lots of space! I have 8 grandchildren and they all love it! ", "doc_count": 2 } ] }, "key": "I just love it! Vans are my favorite vehicle to own! Great for shopping, spacious for hauling large items, very comfortable for road trips! I feel very safe driving my van! ", "doc_count": 2 } ] }, "key": "5392295", "doc_count": 2 }, { "2": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "3": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [] }, "key": "Only one little thing I would change...", "doc_count": 2 } ] }, "key": "635603", "doc_count": 2 }, { "2": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "3": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [] }, "key": "My Pretty Blue Van", "doc_count": 2 } ] }, "key": "906563", "doc_count": 2 } ] } } }

related: https://github.com/opensearch-project/OpenSearch-Dashboards/issues/4995#issuecomment-1981985923

[pslusarz]

@wbeckler

[wbeckler]

Thank you for documenting all this!