Presence for "at-scale" scenarios

[iammeat]

This project looks perfect for a project I'm embarking on, but the "fewer than 100 concurrent subscribers" part of it means that it won't be usable in my situation, since there could be 200, 500 or even 1000 or more users online at a time, with double that offline.

Is there a plan to create a version of this that would use an external database to handle such numbers? Would it involve a lot of work to create such a version?

In any case, thank you for creating and sharing this.

[jondubois]

@iammeat Note that the 100 concurrent subscribers is per-channel; you can add as many channels as you like. You could shard/split up/partition channels in creative ways to make sure that you only track about 100 users at a time (it's kind of unusual to have 1000 user records displayed on the UI at the same time anyway; if you think about it, pagination is a form of sharding so why not shard your channels too?)... Or you could try to track many single-subscriber channels named after specific users so you can check the online/offline status of specific users.

Tracking the online/offline status between thousands of users will use up a lot of messages no matter what (database or no database). Also note that the 100 number is just a guideline; the actual number might be much higher - But I haven't tested.

Other than that, it may be possible to remove hard limits by degrading the accuracy of the service (maybe with a 10-second delay for offline/online notifications); this could be specified as an option in the future but it's not supported right now.

[iammeat]

@jondubois Thanks very much for getting back to me as quickly as you did.

I'll have a think about the best way to approach this, as I see your point about records being displayed. I'm sure there can be a more creative way of handling it, as you mention.

It might be better that I base things around an on-demand search, rather than trying to show everything at once, and splitting it into other channels is a good idea.

I'll have a play around with your code and fork if I come up with anything cool (with PR if you're interested).

[toredash]

I'm looking into this as well, and this plugin doesn't seem to fit my needs with the number of subscribers I'm looking at (+100.000).

Since I don't need realtime presence data, I'm looking at using redis to solve this. I read a good blog post (https://adrianphinney.com/post/68836457654/tracking-active-users-with-redis-and-a-sample-in) that I can recommend.

What I do need is history of number of subscribers to a channel at any give timeslot. More granular = more resources needed, so it really depends on your needs. What I'm currently looking at is for each worker (via worker.getSCServer().clients) extract each clientId and subscriptions and putting them in a redis set:

RPUSH "stats//<ISO 8601 date+time>" clientId [clientId]

This could also be stacked with redis MULTI function, so it seems to be really efficient.

@iammeat Hope this is useful.