Replibyte

Replibyte copied to clipboard

When I create dump from mysql database on GCP cloud storage, it throws below error:

failing query: 'SET @@GLOBAL.GTID_PURGED=/!80000 '+'/ '01442b0f-35da-11eb-8f59-42010ac08011:1-3021439, 1a9e63ac-f93b-11ea-993b-42010ac0800c:1-3522425, 495cffe7-186d-11ea-9edd-42010ac08004:1-3453837, 841d3d46-659a-11eb-a0ba-42010ac08014:1-19157597';' thread 'main' panicked at 'TokenizerError { message: "Unterminated string literal", line: 1, col: 11 }', dump-parser/src/mysql/mod.rs:708:13 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

Can someone help me to get rid of this error?

Hi @paragkapoorcs ,

Does

SET @@GLOBAL.GTID_PURGED=/!80000 '+'/ '01442b0f-35da-11eb-8f59-42010ac08011:1-3021439,
1a9e63ac-f93b-11ea-993b-42010ac0800c:1-3522425,
495cffe7-186d-11ea-9edd-42010ac08004:1-3453837,
841d3d46-659a-11eb-a0ba-42010ac08014:1-19157597';

is the complete query? Otherwise, can you give me the complete one so that I try to reproduce the error?

Romaric.

I have run the below .yaml script with all the variables:

source: connection_uri: mysql://user:password@hostname:port/databasename transformers: - database: public table: table-name columns: - name: column-name transformer_name: random datastore: gcp: bucket: your-bucket-name region: bucket-region access_key: $GS_ACCESS_KEY secret: $GS_SECRET destination: connection_uri: mysql://user:password@hostname:port/databasename

When I perform this code without transformer, it is working fine for me but when i perform with transformer, it throws below error:

failing query: 'SET @@GLOBAL.GTID_PURGED=/!80000 '+'/ '01442b0f-35da-11eb-8f59-42010ac08011:1-3021439, 1a9e63ac-f93b-11ea-993b-42010ac0800c:1-3522425, 495cffe7-186d-11ea-9edd-42010ac08004:1-3453837, 841d3d46-659a-11eb-a0ba-42010ac08014:1-19157597';' thread 'main' panicked at 'TokenizerError { message: "Unterminated string literal", line: 1, col: 11 }', dump-parser/src/mysql/mod.rs:708:13 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

Here is the screenshot of error:

Thank you I am going to take a look

Hi there, I am getting the same error message on various databases/tables.

my deduction is that when a ; [semicolon] is used in a field, that throws the Tokenizer.

I am using mysql.

Also coming across this when a ; is used inside a column value with MySQL.

Seems problem is inside dump-parser/src/utils.rs in list_statements.

Here is a test input (a JSON array containing a single string ["A'B; C'D E'F;"]) that causes a failure:

let s = list_statements(
    r#"INSERT INTO public.toto (value) VALUES ('[\"A\'B; C\'D E\'F;\"]');"#,
);

So either list_statements needs to change or there needs to be different escaping before calling this calling function as far as I can tell.

To give a clear comparison: The insert value from MySQL is passed as '[\\\"A\\'B; C\\'D E\\'F;\\\"]' The same insert value from Postgres is passed as '[\"A''B; C''D E''F;\"]'

I can see a fix is merged to try to fix this issue, but I'm on the latest version (0.9.6) and still get an issue with semicolons

Here is the SQL that triggers it:

INSERT INTO `contact` (....) VALUES 
(3434343,NULL,NULL,NULL,'2021-12-29 01:16:57',NULL,NULL,'[email protected]','CREATED',NULL,'John 842631616',NULL,'Doe 910125632',NULL,'+1111111111',NULL,NULL,1,'1 Test st;'

The semicolon is right at the end, maybe the fix didn't capture it?

+1

+1

Thanks for your feedback - I will provide a fix. I am also looking at changing the way we parse queries more appropriately. Our handwritten parser starts to be complex to maintain.