reactivesearch
reactivesearch copied to clipboard
appbaseio
Infinite scroll creates duplicate records
Issue Type: Bug
Platform: Web
Description: When using infinite scroll, if you scroll down past the first page or so it shows new results but loads duplicate hits. It appears the duplicates are the last hit of the previous "page". In addition to showing duplicates, it also throws a React error:
Warning: Encountered two children with the same key, `{id}`. Keys should be unique so that
components maintain their identity across updates. Non-unique keys may cause children to be
duplicated and/or omitted — the behavior is unsupported and could change in a future version.
Screenshots: Video demo of issue
Reactivesearch version: x.y.z 3.0.0-beta.10
Browser: All
One way to address this is to add support for Elasticsearch's "search_after" field: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-search-after.html. Perhaps there could be an option to choose between "from" and "search_after" based paging for infinite scrolling.
I can see this adding value to the state of things only if the results are getting updated in the real time. Do you see any other benefit of using search_after?
It depends what you mean.
If you mean that the data in the Elasticsearch index is being updated in real time, which is what causes duplicate items to appear by items being bumped up a position in between the client loading one page and the next, then I think this benefit is justified on it's own.
If you mean that the data on the client is being updated in real time, such that data that has already been loaded and displayed on the screen will have items updated, removed, or inserted, then it depends on the use case. There are many situations where it's required to not show duplicates, but not necessarily to update data that's already been rendered to the screen.
There's no universal best way to handle search results where the underlying data is updated while the user is scrolling. Each situation requires balancing tradeoffs, and supporting "search_after" paging provides another commonly desired option.
Hey @jrock17
I was not able to replicate this issue. I tried with multiple datasets and different sizes of ReactiveList. Can you confirm if this is happening in the latest version ?
A user may see duplicate items when a previous item is added after one page is loaded but before the next. This can occur not only when a new item is inserted into the index, but also when an existing item that didn't match the current search criteria is modified such that it does, or when an existing item that would have appeared later based on the current sorting is modified such that it appears earlier.
Consider search results using infinite scrolling for first names sorted alphabetically, with an initial total of 5 items and a page size of 2. When the user loads the first and second page of results, they will see ["Alice", "Bob", "Charlie", "David"]. Imagine that then, a new item "Andrew" is added to the underlying dataset, which falls between "Alice" and "Bob" when sorted alphabetically. When the user then scrolls to request the third page of results, they will see ["David", "Edward"], as they are now the fifth and sixth items, even though "David" was already displayed on the second page.
Possibly related to this re-rendering issue (still a problem)?
By the way we observe duplicate key errors with multi-list when the app data does not change. Search filters changes typically trigger that (Gatsby SSR setup similar to this one).
A user may see duplicate items when a previous item is added after one page is loaded but before the next. This can occur not only when a new item is inserted into the index, but also when an existing item that didn't match the current search criteria is modified such that it does, or when an existing item that would have appeared later based on the current sorting is modified such that it appears earlier.
Consider search results using infinite scrolling for first names sorted alphabetically, with an initial total of 5 items and a page size of 2. When the user loads the first and second page of results, they will see ["Alice", "Bob", "Charlie", "David"]. Imagine that then, a new item "Andrew" is added to the underlying dataset, which falls between "Alice" and "Bob" when sorted alphabetically. When the user then scrolls to request the third page of results, they will see ["David", "Edward"], as they are now the fifth and sixth items, even though "David" was already displayed on the second page.
maybe you found a solution to this problem? I faced the same problem only on another project. UPD: find solution
