Get the encode characters issue when run osc_cli command

[lynch0227]

Hi Guys,

I want to create a comment using mandarin language, but got the below errors: UnicodeEncodeError: 'latin-1' codec can't encode characters in position 238-242: ordinal not in range(256) Could you fix issue to support the multiple language?

And I have another question, I have to specify the COLLATE 'utf8mb4_general_ci' attribute in column level, but I got the parse sql errors: ERROR 2017-05-25 14:37:13.314 INVALID_SYNTAX: Fail to parse: foo.sql Line: enroll_reason VARCHAR(1024) NULL DEFAULT NULL COLLATE 'utf8mb4_general_ci',, Column: 57 Failed to parse SQL, unsupported syntax: Expected ")" (at char 634), (line:17, col:57) Most likely is not a valid CREATE TABLE sql. Please make sure it has correct syntax and can be executed in MySQL: Line: enroll_reason VARCHAR(1024) NULL DEFAULT NULL COLLATE 'utf8mb4_general_ci',, Column: 57 Failed to parse SQL, unsupported syntax: Expected ")" (at char 634), (line:17, col:57)

Could you please fix it?

Thank you.

[cenalulu]

@lynch0227 Thanks for the report!

I'll need more information to reproduce the UnicodeEncodeError. Basically it will be super helpful, if you could provide following information:

• An CREATE TABLE example
• What you specified for --charset option on CLI
• The file encoding of the ddl file

As for the parse error, I'm able to reproduce it successfully. Since it's a separate issue, I filed https://github.com/facebookincubator/OnlineSchemaChange/issues/13 to get it fixed.

[lynch0227]

@cenalulu Here is the create table statement: CREATE TABLE t ( id BIGINT(20) NOT NULL AUTO_INCREMENT, enroll_reason varchar(1024) CHARACTER SET utf8mb4 DEFAULT NULL COMMENT '推荐内容', enroll_time DATETIME NULL DEFAULT NULL COMMENT '报名时间', enroll_reason VARCHAR(1024) NULL DEFAULT NULL, enroll_user_type TINYINT(4) NULL DEFAULT NULL, create_time DATETIME NOT NULL, last_update_time TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
is_test TINYINT(4) NULL DEFAULT 1 COMMENT '测试标记', PRIMARY KEY (id) ) COLLATE='utf8_general_ci' ENGINE=InnoDB

And I don't have specify the --charset option on CLI. The ddl file is 'UTF-8 Unicode text'

[cenalulu]

@lynch0227 Could you try running osc_cli with --charset=utf8. If the schema contains multi-bytes characters osc is known to have that issue. I'll create a follow up to make the error message more self explained and suggesting the right approach.