python-mysql-replication icon indicating copy to clipboard operation
python-mysql-replication copied to clipboard

upgrade and improve perfermance

Open YaoDongJi opened this issue 1 year ago • 12 comments

Version

Please specify the versions you are using. Exact version numbers are preferred.

  • Pymyrepl (e.g., 1.0.2): 0.30
  • OS (e.g., Ubuntu 18.04): Debian
  • Database and version (Remove unnecessary options): Mysql 5.7
    • MySQL:
    • MariaDB:
    • Percona:

Symptoms

We are currently using 0.31 as our production version, but we have encountered a performance bottleneck with a processing speed of 60m/s. Is the latest version compatible with version 0.31? Can we improve performance by upgrading the version?

YaoDongJi avatar Jan 27 '24 06:01 YaoDongJi

@YaoDongJi If you use Mysql 5.7version Use 0.45.1 version

sean-k1 avatar Jan 29 '24 05:01 sean-k1

您好,我已收到邮件 。                                                     收件人:姚东济

YaoDongJi avatar Jan 29 '24 05:01 YaoDongJi

You can try using PyPy instead of the standard python interpreter this is normally faster.

Le lun. 29 janv. 2024 à 06:22, YaoDongJi @.***> a écrit :

您好,我已收到邮件 。 收件人:姚东济

— Reply to this email directly, view it on GitHub https://github.com/julien-duponchelle/python-mysql-replication/issues/603#issuecomment-1913983745, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACUKXIXEDNFDJL3ZQHRROLYQ4W2HAVCNFSM6AAAAABCNECBVCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTHE4DGNZUGU . You are receiving this because you are subscribed to this thread.Message ID: <julien-duponchelle/python-mysql-replication/issues/603/1913983745@ github.com>

julien-duponchelle avatar Jan 29 '24 06:01 julien-duponchelle

@YaoDongJi If you use Mysql 5.7version Use 0.45.1 version

thanks, we will try this version

YaoDongJi avatar Jan 30 '24 01:01 YaoDongJi

You can try using PyPy instead of the standard python interpreter this is normally faster. Le lun. 29 janv. 2024 à 06:22, YaoDongJi @.***> a écrit : 您好,我已收到邮件 。 收件人:姚东济 — Reply to this email directly, view it on GitHub <#603 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACUKXIXEDNFDJL3ZQHRROLYQ4W2HAVCNFSM6AAAAABCNECBVCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTHE4DGNZUGU . You are receiving this because you are subscribed to this thread.Message ID: <julien-duponchelle/python-mysql-replication/issues/603/1913983745@ github.com>

We are currently using pypy3.7. Currently, we have observed a maximum of 10000 tps per second, corresponding to approximately 10MB of data. We would like to ask for some observation indicators that can be used as a reference to help us confirm whether there is still room for performance improvement.

YaoDongJi avatar Jan 30 '24 01:01 YaoDongJi

@YaoDongJi we don't maintain this for mysql5.7 anymore, but I can put this feature in 0.45.2 if you want ?

sean-k1 avatar Jan 30 '24 06:01 sean-k1

n 0.45.2

Yes, performance improvement is really needed. Thank you very much for your help. We will test the performance of this version after merging this feature.

YaoDongJi avatar Jan 30 '24 09:01 YaoDongJi

Could you please help us merge this feature into version 0.30? I checked and found that we are using version 0.30, and I compared the performance of 0.45.1 and version 0.30, and found that 0.30 will perform better than 0.45.1, thanks.

YaoDongJi avatar Feb 02 '24 01:02 YaoDongJi

@YaoDongJi Need 0.30.1 or 0.31.1?

sean-k1 avatar Feb 03 '24 05:02 sean-k1

@YaoDongJi Need 0.30.1 or 0.31.1?

0.30.1 would be better.

YaoDongJi avatar Feb 04 '24 00:02 YaoDongJi

@YaoDongJi released 0.30.1 Let me know if you have any problems

sean-k1 avatar Feb 08 '24 08:02 sean-k1

@YaoDongJi released 0.30.1 Let me know if you have any problems

We will test this soon,thanks

YaoDongJi avatar Feb 18 '24 02:02 YaoDongJi