metaflow-service icon indicating copy to clipboard operation
metaflow-service copied to clipboard

Change websocket cleanup performance from O(n^2) to O(n).

Open marshallbrekka opened this issue 1 year ago • 2 comments

Move subscription tracking to a dict keyed by the underlying websocket object.

Changes the cleanup from two nested for loops (the 2nd being the unsubscribe_from method), to a single loop with (the common case) deleting a single key from a dict.

We found that with a sufficient number of disconnected clients, and long gaps between database activity, the cleanup could end up taking multiple hours. This would cause the python web service to become unresponsive.

marshallbrekka avatar Jun 17 '24 19:06 marshallbrekka

lgtm! we can ship this once @saikonen approves

savingoyal avatar Jun 17 '24 19:06 savingoyal

@savingoyal @saikonen is there any update here, anything I can do to help get this upstreamed? thanks!

marshallbrekka avatar Aug 20 '24 17:08 marshallbrekka