Error starting change stream

[Jap8nted]

Hi,

Our Prod Mongo is a running on version 3.6.0. We currently have a synchronisation service that is reading from the change stream to update index in Elastic search using monstache (https://github.com/rwynn/monstache). After we made some update on our data we got this:

ERROR 2021/05/03 16:00:59 Error starting change stream. Will retry: CappedPositionLost: CollectionScan died due to position in capped collection being deleted.

And now I am not able to enable the synchronisation again.

Do you know a way I can enable the synchronisation process?

Thank you in advance

[rwynn]

Hi @Jap8nted are you able to restart monstache with direct-read-namespaces configured to do a full sync of the index? This should also restart the change stream if you are using change-stream-namespaces also.

[Jap8nted]

Hi @rwynn, sorry for the late answer. I restarted monstache successfully. The issue was that the change Stream had already expired. We modified it and run a whole sync process to recover the service.

However we have had problems with monstache as it happens to stop running without any error or log. It just stops. It seems to happen when too many documents are being synced. Is there any way of debugging this? it is a difficult to reproduce error and just happens in PROD. Our QA environment has never suffered from this issue.

Shall I open a new issue for that?

Thanks!