python-mysql-replication
python-mysql-replication copied to clipboard
improve performance and documentation, skip the first 2 events after reconnected
* sleep 5s after disconnected accidently and skip the first 2
events after reconnected
* symplify BinLogStreamReader.fetchone
* improve documentation of BinLogStreamReader
* some cleanup and compacting works
* tests/test_data_type.py: fix syntax errors
* tests/test_basic.py: dynamic get an end position instead a fixed one
@woods-chen
I love the * symplify BinLogStreamReader.fetchone
feature.
I think it will greatly improve the performance of the module.
(though I haven't tested yet)
However, I'm a little concerned about * sleep 5s after disconnected accidently and skip the first 2 events after reconnected
feature.
The very first event the module receives upon connection is RotateEvent
, which triggers table_map
to be flushed:
https://github.com/noplay/python-mysql-replication/blob/7fd706d1686da90cdb3991279c764f057e24a1f6/pymysqlreplication/binlogstream.py#L507-L519
Consider a scenario where reboot of MySQL caused disconnection.
mysql-replication will sleep for 5 seconds and skip the first RotateEvent
.
Then move on without flushing table_map
to be filled with table ids that is not valid any more after MySQL reboot.
Nice work with the optimization, but I think we should keep parsing the first RotateEvent
for maintaining valid table_map
.
@dongwook-chan Thank you for correcting me. I should have read the previous codes and comments in more detail before committing. I only considered the situation that the network is disconnected, and sleeping 5 seconds is also for waiting for the network to be resumed. And I pushed another commit which deleted those lines .
@woods-chen Thank you for the correction.