connectors icon indicating copy to clipboard operation
connectors copied to clipboard

Sync failure: cursor id xxxxxxxxxx not found

Open ppf2 opened this issue 1 year ago • 0 comments

When running MongoDB connector with a ML training model (e.g. ELSER), the added time spent on running the data against the training model appears to be causing the MongoDB cursor to timeout:

image

This results in a failure to sync with a singular log entry showing a critical error:

image

If I add sync rules to limit the amount of data that gets processed by the training model, the job runs fine then.

We are looking into improving observability in this area via https://github.com/elastic/connectors-python/issues/961. Just filing in case others are running into this. If this ends up being a cursor timeout issue, it can be helpful to remove/extend the cursor timeout.

ppf2 avatar May 31 '23 00:05 ppf2