polar icon indicating copy to clipboard operation
polar copied to clipboard

Implement EventStream publish as a queue that's flushed *after* request completion

Open frankie567 opened this issue 1 year ago • 0 comments

Currently, publish sends the data immediately to Redis, which might provoke unpredicted behaviors because the request didn't committed the changes to the database yet, or something might fail and error afterwards.

We should revamp it so it works similar to enqueue_job for worker tasks: locally accumulate the messages in a queue and flush it after the request has completed.

In some cases, the current behavior might be desired (e.g. when syncing several GitHub issues inside a single task). For this case, we may have a flag to tell the function to immediately flush the queue.

frankie567 avatar Oct 03 '24 09:10 frankie567