Fix reindex problem

[salihozkara]

Description

Resolves https://github.com/volosoft/vs-internal/issues/4175

• As the count of documents increases, the problems with memory increase. Pagination has been added to prevent this.
• When indexing, the last cached document should be indexed as unique