Replibyte icon indicating copy to clipboard operation
Replibyte copied to clipboard

MySQL why use --skip-extended-insert?

Open moisesrodriguez opened this issue 2 years ago • 4 comments

Looking at how mysqldump is used, I noticed the flag --skip-extended-insert. Why write INSERT statements using one-row syntax that includes only one record in one statement, if it will be much much slower than having multiple records in one INSERT statement? Specially when importing a lot of data.

moisesrodriguez avatar May 17 '22 18:05 moisesrodriguez

Hi @moisesrodriguez This flag was added so that the final dump is as close as possible to that of PostgreSQL, the reason being that it was easier to parse without major change in the dump parser for assuring a compatibility with MySQL. Indeed, this has a performance impact but this will be changed in a future release, the objective being to have the most efficient tool possible.

Contributions are welcome to improve the tool.

fabriceclementz avatar May 17 '22 19:05 fabriceclementz

Hey @moisesrodriguez (happy to see you here :)) It's related to this https://github.com/Qovery/Replibyte/issues/36 Once we'll fix it for PSQL, it would be easy to port that change to MySQL. As @fabriceclementz said, feel free to contribute if you have some time. I don't think it's a big deal, to be honest, just a matter of 2 to 3 hours of work (tests included).

evoxmusic avatar May 17 '22 19:05 evoxmusic

Hi @evoxmusic This is the next issue I will try to tackle, As COPY for Postgres, This will considerably improve the performance of Replibyte, I let you know my progress.

fabriceclementz avatar Aug 07 '22 05:08 fabriceclementz

Thank you @fabriceclementz - it's definitely a must have improvement to increase restore performances. I am also looking how we can better bench and watch Replibyte performances

evoxmusic avatar Aug 07 '22 06:08 evoxmusic