[Bug][GitHub] Task error 3988 when collecting data from GitHub Enterprise

[gabytal]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

Dear Devlake developers, We got this error for a Github task when tried to run a pipeline.

subtask extractApiPullRequests ended unexpectedly Wraps: (2) Error 3988 (HY000): Conversion from collation utf8mb4 0900 ai ci into utf8mb3 general ci impossible for parameter (500) Wraps: (3) Error 3988 (HY000): Conversion from collation utf8mb4 0900 ai ci into utf&mb3 general ci impossible tor parameter Error tvpes: (1) *hintdetail.withDetail (2) *hintdetail.withDetail(3) *mysql. MvSQLError

Couldn't find anything helpful from logs.

devlake:v0.18.0 database: mysql

Thanks. Appreciate any kind of help.

What do you expect to happen

Expect for the task to be completed successfully

How to reproduce

1. Browse to https://devlake/projects/testproj?tab=status
2. Press "Collect All Data"

Anything else

it happens only for some Github repos, some other Github tasks are OK

Version

v0.18.0

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[mkaufmaner]

This error is intermittent for some repositories in GitHub enterprise.

[d4x1]

@gabytal What's your MySQL version and have you upgrade it from a lower version?

[abeizn]

Can you check the character set? SELECT character_set_name, collation_name FROM information_schema.columns WHERE table_schema = 'lake';

And please set CHARACTER_SET_NAME="utf8mb4" and COLLATION_NAME="utf8mb4_bin". related to https://github.com/apache/incubator-devlake/issues/6205

[cicdguy]

Sorry, reposting this as this can be inherently solved within the migration scripts that DevLake uses to set up the schema - can these simply be added to the migration scripts? That way users do not need to explicitly set this all the time.

[klesh]

@cicdguy I agree, just need to figure out a solution for MySQL while not affecting Pg

[mkaufmaner]

Can you check the character set? SELECT character_set_name, collation_name FROM information_schema.columns WHERE table_schema = 'lake';

And please set CHARACTER_SET_NAME="utf8mb4" and COLLATION_NAME="utf8mb4_bin". related to #6205

@abeizn This worked. Also had to run a script to change the existing charset and collation for all tables.

-- To get the SQL to run for all devlake tables
SELECT CONCAT('ALTER TABLE ', tbl.TABLE_SCHEMA, '.', tbl.TABLE_NAME, ' CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;') FROM information_schema.TABLES
 tbl WHERE tbl.TABLE_SCHEMA = 'devlake'

[github-actions[bot]]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[michaelawarren]

I'm seeing this issue too. Will try the work around. would be nice if the DB setup/migration did this automatically.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.