nchan icon indicating copy to clipboard operation
nchan copied to clipboard

nchan <=> redis pub/sub?

Open hellochenwang opened this issue 9 years ago • 10 comments

Hi Leo, would it be useful/possible if nchan publishes and subscribes to redis pub/sub channels? essentially nchan provides a websocket interface to redis pub/sub.

hellochenwang avatar Sep 10 '16 17:09 hellochenwang

You're not the first person to ask about this. It's entirely possible, and actually not hard to implement. My question for you is why do you want this feature? What problems would this help you solve?

slact avatar Sep 10 '16 20:09 slact

I'm building an IoT project, redis pubsub is the messaging bus/IPC. I am planning to build a web interface that can have access to the messaging bus.

There are some nodejs implementations to do this, but I think nchan would have much better performance.

hellochenwang avatar Sep 10 '16 20:09 hellochenwang

Ok, I understand.

The biggest challenge for me with this feature is how to design its configuration. There are some things that cannot be avoided which will make it quirky at best, and unintuitive at worst. Most importantly, this feature will need to be configured on subscriber locations, which is weird. It would look something like this:

http {

   server {
      listen 80;

      location ~ /subscribe/(.*)$ {
        nchan_subscriber;
        nchan_channel_id $1;
        nchan_upstream_message_source /publish/$1;
      }

      location ~ /publish/(.*)$ {
        internal;
        nchan_channel_id $1;
        nchan_publisher redis-pubsub;
        nchan_redis_url "redis://localhost:6379/0";
        nchan_publisher_redis_pubsub_key "channel_$1";
      }

   } 
}

Is that configuration you'd understand just by looking at it?

slact avatar Sep 10 '16 21:09 slact

nchan_publisher redis-pubsub; This line makes me think all the nchan_channel_ids would be mapped to redis pubsub channel ids.

nchan_publisher_redis_pubsub_key "channel_$1"; This line makes me think nchan_channel_id "$1" maps to redis pubsub channel "channel_$1".

They both make sense to me. The 2nd one seems more flexible.

hellochenwang avatar Sep 10 '16 21:09 hellochenwang

for the subscriber, possible to have configurations look like below?

nchan_subscriber redis-pubsub;

or

nchan_subscriber_redis_pubsub_key "channel_$1";

hellochenwang avatar Sep 10 '16 21:09 hellochenwang

for the subscriber, possible to have configurations look like below? [...]

Unfortunately not. The publisher needs to be configured with a location block, so that's why nchan_upstream_message_source /publish/$1; refers to the internal publisher location block.

nchan_publisher_redis_pubsub_key "channel_$1";
This line makes me think nchan_channel_id "$1" maps to redis pubsub channel "channel_$1".

That's correct.

Anyway, I'll think about how to implement this. I've had other requests for a similar feature subscribing to upstream Nchan servers, and they will need similar methods of configuration and some similar code work. I'll update this when I come up with a satisfactory solution.

slact avatar Sep 10 '16 22:09 slact

If Nchan has an API like hiredis, then subscribing to upstream Nchan servers would be easier.

Thanks for the awesome Nchan!

hellochenwang avatar Sep 10 '16 22:09 hellochenwang

this could be freaking awesome !!

SunChero avatar Dec 16 '16 21:12 SunChero

resurrecting this to ask: is there any interest in sponsoring this feature?

slact avatar Dec 02 '17 01:12 slact

@slact can you post a newer version of the same example that you posted before? looks like some of the config values don't apply anymore

santiagoalmeidabolannos avatar Jul 12 '21 15:07 santiagoalmeidabolannos