mydumper icon indicating copy to clipboard operation
mydumper copied to clipboard

[BUG] myloader break not response when error occours

Open firdibasri opened this issue 2 years ago • 2 comments

When I use myloader and looping error occours (its from my mistake), I try to stop it by push ctrl+C but the errors still looping until kill the process (kill command)

Command executed:

  • mydumper with all the parameters mydumper -u theUser -p secret -P 6033 -h ip_source --database=theDB --tables-list=table1,table2--outputdir=outdir --threads=4 --compress
  • myloader with all the parameters myloader -u theUser -p secret -P 6033 -h ip_destination --database=theOtherDBname --directory=outdir --queries-per-transaction=50000 --threads=4 --compress-protocol --verbose=3

mydumper and myloader version: 0.12.1

Expected behavior Should if the looping error occours, and I push Ctrl+C, it shows confirmation as usually "Ctrl+c detected! Are you sure you want to cancel(Y/N)?"

Environment

  • OS version: Ubuntu 20.04
  • MyDumper version: 0.12.1
  • MariaDB: 10.7.x

Additional context Sample error occours:

** (myloader:246777): CRITICAL **: 19:10:38.572: Error occours between lines: 1554141 and 1560776 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

** (myloader:246777): CRITICAL **: 19:10:38.727: Error occours between lines: 1560777 and 1567630 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

** (myloader:246777): CRITICAL **: 19:10:38.881: Error occours between lines: 1567631 and 1574338 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

** (myloader:246777): CRITICAL **: 19:10:39.037: Error occours between lines: 1574339 and 1580986 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

** (myloader:246777): CRITICAL **: 19:10:39.243: Error occours between lines: 1580987 and 1587844 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1 ^C ** (myloader:246777): CRITICAL **: 19:10:39.401: Ctrl+c detected! Are you sure you want to cancel(Y/N)?

** (myloader:246777): CRITICAL **: 19:10:39.450: Error occours between lines: 1587845 and 1594498 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

** (myloader:246777): CRITICAL **: 19:10:39.645: Error occours between lines: 1594499 and 1601108 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

** (myloader:246777): CRITICAL **: 19:10:39.832: Error occours between lines: 1601109 and 1607948 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

** (myloader:246777): CRITICAL **: 19:10:40.019: Error occours between lines: 1607949 and 1614826 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

** (myloader:246777): CRITICAL **: 19:10:40.197: Error occours between lines: 1614827 and 1621397 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

** (myloader:246777): CRITICAL **: 19:10:40.372: Error occours between lines: 1621398 and 1628011 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

** (myloader:246777): CRITICAL **: 19:10:40.544: Error occours between lines: 1628012 and 1634783 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

** (myloader:246777): CRITICAL **: 19:10:40.710: Error occours between lines: 1634784 and 1641415 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

** (myloader:246777): CRITICAL **: 19:10:40.879: Error occours between lines: 1641416 and 1648246 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

** (myloader:246777): CRITICAL **: 19:10:41.048: Error occours between lines: 1648247 and 1655007 on file theDB.table1.00000.sql.gz: Column count doesn't match value count at row 1

firdibasri avatar Mar 23 '22 12:03 firdibasri

Hi @firdibasri , Currently the pause/resume functionality is at file level, not at statement level. This means that myloader needs to complete the file to know that you want to pause. I will be working on better implementation in future releases.

davidducos avatar Mar 23 '22 12:03 davidducos

What we should be implementing, is that on a second ctrl+c, we must kill mydumper/myloader.

davidducos avatar May 30 '22 17:05 davidducos

I tried to catch the second ctrl+c but I was not able to do it. During my research I found/realize that you can simply ctrl+\ to kill the process that is what you want.

davidducos avatar Sep 12 '22 13:09 davidducos