aembke
Channel Lag issue
Fred version - 9.1.2 - REPLICAS Features enabled Redis version - 7.2.5 JEMALLOC Platform - LINUX Debian Bullseye Deployment type - cluster
My cluster setup consists of 4 servers. Each 3 masters and to each master 1 slave.
12 masters - 12 slaves in total.
Describe the bug Not sure if it's a bug, but I think it is redis-cli doesn't suffer from this. Fred does. After a while it says Error channel lagged by 5559. This comes from using the RX.
To Reproduce Steps to reproduce the behavior: Just heavily fast send to a channel and wait a couple of seconds
Following code:
use fred::{
prelude::{ClientLike, EventInterface, PubsubInterface, RedisClient},
types::RedisConfig,
};
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let config = RedisConfig::from_url("redis-cluster://masked:7002?node=masked:7002&node=masked:7005&node=masked:7008&node=masked:7011")?;
let client = RedisClient::new(config, None, None, None);
client.init().await?;
client.ssubscribe("channel").await?;
let mut rx = client.message_rx();
loop {
let message = rx.recv().await?;
println!("Message: {:?}", message);
}
Ok(())
}
Should get an error like this:
Logs
Error: channel lagged by 5559
(If possible set RUST_LOG=fred=trace and run with --features debug-ids)
Hi @CR7273868,
That's an indication that the broadcast channel is receiving messages faster than they're being processed on the receiving end. You can change the buffer size of the broadcast channels with the broadcast_channel_capacity configuration option like this:
https://github.com/aembke/fred.rs/blob/f798232966ff6ee75ad60c09f42bef67f440a9ed/examples/misc.rs#L16-L17
The default is 32, so you'll probably want to increase that quite a bit.
