How to choose the right intervals for the "Keep Alive" and "Heartbeat" features

[denisahearn]

Hi,

We're looking to use this project within JRuby to detect changes in MySQL. We have a proof-of-concept up and running using BinaryLogClient#connect. Based on the available documentation (i.e. https://github.com/shyiko/mysql-binlog-connector-java/blob/master/readme.md), we expected this method to block indefinitely, however we've found that it returns (i.e. stops blocking) after a certain period of time. After going back to the documentation again and not finding any answers, we stumbled across https://github.com/shyiko/mysql-binlog-connector-java/issues/102, which led us to https://github.com/shyiko/mysql-binlog-connector-java/issues/118 and ultimately to https://github.com/shyiko/mysql-binlog-connector-java/commit/a84200ee21d1313e35a7fba454c59d4545050636#diff-7addc4e3eed6e9254e088abc015ac8adR393.

We've now configured BinaryLogClient via setKeepAliveInterval and setHeartbeatInterval, and that seems to have solved the problem with BinaryLogClient#connect returning. We're using 60,000 milliseconds for the keep alive interval, and 10,000 milliseconds for the heartbeat interval, however, we have no real basis for choosing those numbers. 60,000 milliseconds appears to be the default keep alive interval, and 10,000 milliseconds was chosen as the heartbeat interval to ensure that we're using a value less than the keep alive interval.

What consideration/criteria goes into choosing the proper intervals for those settings? Are there general rule-of-thumb values we should be using? What characteristics of an application would cause someone to fine tune those settings?

Also, I know we would've been saved some time had the documentation mentioned the "Keep Alive" and "Heartbeat" features when talking about BinaryLogClient. Perhaps just adding a blurb about each feature to https://github.com/shyiko/mysql-binlog-connector-java#tapping-into-mysql-replication-stream would save other newbies some time.

Thanks for all your hard work on this project.

Best regards, Denis

[shyiko]

Hi.

Choice of keepAliveInterval/heartbeatInterval depends on how fast you want broken connection to be detected & restored, network latency, etc. Think about keepAliveInterval as N x heartbeatInterval, where N is a maximium number of HEARTBEAT events that can be lost (consequently) before connection must be re-established. For example, keepAliveInterval=15000(ms) & heartbeatInterval=5000(ms) results in HEARTBEAT event every 5 seconds and maximum broken-connection period of 15s (in other words - "reconnect if ~3 (5 seconds apart) HEARTBEAT events were lost") (normally connection drop is detected immediately and this is really just a fail safe for a number of not-so-normal-but-still-frequently-seen other cases). Does it make sense?

Regarding the docs: you are absolutely right. I should have mentioned HEARTBEAT events a long time ago. Thank you for pointing it out!

[denisahearn]

Hi,

Yes, your explanation makes perfect sense.

Thanks again, Denis

[shyiko]

Also, I know we would've been saved some time had the documentation mentioned the "Keep Alive" and "Heartbeat" features when talking about BinaryLogClient. Perhaps just adding a blurb about each feature to https://github.com/shyiko/mysql-binlog-connector-java#tapping-into-mysql-replication-stream would save other newbies some time.

(keeping this ticket open until readme is updated)