meilisync icon indicating copy to clipboard operation
meilisync copied to clipboard

Meilisync loses event when both create and update occur for the same primary key in an interval

Open rishabhc32 opened this issue 10 months ago • 2 comments

If I have an interval of, let's suppose, 10 seconds and insert size of 1000 documents, and within that interval, a create event is first received followed by an update event for the same primary key, the current implementation of Meilisync's event buffer leads to the create event being overwritten by the update event.

https://github.com/long2ice/meilisync/blob/052c4de9ab4e7ed91285b981c7ab5beeb03e90ec/meilisync/event.py#L10-L13

self._events[sync][pk] = event here the event will get overwritten.

A potential solution could involve modifying the event handling logic to append events rather than overwrite them, for instance:

self._events[sync][pk].append(event)

Or we could just do:

self._events[sync].append(event)

rishabhc32 avatar Mar 27 '24 20:03 rishabhc32

@long2ice Would you be interested in a PR where a solution like self._events[sync][pk].append(event) is implemented? This will preserve the order of the events on a primary key.

rishabhc32 avatar May 31 '24 20:05 rishabhc32

I had included my fix for this on https://github.com/long2ice/meilisync/pull/103

KDMW-IO avatar Aug 09 '24 09:08 KDMW-IO