make pubsub subRefsMap use maps and sets

[tjmehta]

Simplify pubsub subscription-id state by using maps and sets

[davidyaha]

Hey @tjmehta, Thanks for the PR! I'd love to merge this but can you fix the failing tests? Thanks ahead!

[tjmehta]

cool was wondering if y'all would even want this.

I went off the deep-end and wrote my own module bc I was paranoid about memoryleaks https://github.com/tjmehta/graphql-ioredis-subscriptions (would you want me to bring in any of this functionality, like promise queues by triggerName?). I am still seeing memory leaks related to subscriptions in my application and I cannot repro in development. I even found some potential leaks in https://github.com/tjmehta/subscriptions-transport-ws/commit/6d425ba694939df4ee6ea32c22e1072dd06f82f5. Would love some tips if you've had issues with memory leaks around with graphql subscriptions.

[thecodeboss]

@davidyaha this PR should be passing tests now and ready to merge. For some context, my company experienced issues where we had around ~50k subscribers, and this was causing the array operations to be very slow when every a new subscriber joined, or one dropped off. For example, adding an ID to the array using the spread operator

[...refs, id]

was causing an array of ~50k elements to get fully copied. In our setup, these expensive copies were happening multiple times per second, causing a noticeable performance hit. The maps/sets remove the need for these expensive array operations.

[CutmanCometh]

@davidyaha Any chance you'd consider merging this? Can help out where needed.

[davidyaha]

Thanks, @tjmehta @thecodeboss for the PR! and Thanks @CutmanCometh for the willingness to help.