DBD::mysql does not respect UTF-8 flag on data being written [rt.cpan.org #60987]

[mbeijen]

trafficstars

Migrated from rt.cpan.org#60987 (status was 'open')

Requestors:

• [email protected]

From [email protected] on 2010-09-02 15:06:57:

DBD::mysql doesn't respect the UTF8 flag on strings supplied, either as 
SQL, or as bound parameters.  This can cause the wrong data to be 
written to your database, whether or not you pass mysql_enable_utf8=1, 
and whether you table charset is latin1 or utf8.

This test demonstrates the problem, http://github.com/davel/DBD-
mysql/commit/8695068684919d9751550a88aaf22df961eb5359

If mysql_enable_utf8 is unset, all strings with the SvUTF8 flag set 
should be run through bytes_from_utf8.

If mysql_enable_utf8 is set, most strings with SvUTF8 flag unset want to 
be run through bytes_to_utf8.  I say most, as if you're populating a 
BLOB, you don't want to upgrade your string to UTF-8, which DBD::mysql 
probably can't determine.  Still, behaviour which doesn't depend on an 
obscure internal flag would be a step forward!

This report may be a more general bug from the one in RT#25590.  I don't 
think it's the same problem as RT#53130, which is concerned with data 
being read.

Kind regards,
Dave Lambley

From [email protected] on 2016-10-22 15:10:37:

On Å tv Sep 02 11:06:57 2010, [email protected] wrote:
> DBD::mysql doesn't respect the UTF8 flag on strings supplied, either as 
> SQL, or as bound parameters.  This can cause the wrong data to be 
> written to your database, whether or not you pass mysql_enable_utf8=1, 
> and whether you table charset is latin1 or utf8.
> 
> This test demonstrates the problem, http://github.com/davel/DBD-
> mysql/commit/8695068684919d9751550a88aaf22df961eb5359
> 
> If mysql_enable_utf8 is unset, all strings with the SvUTF8 flag set 
> should be run through bytes_from_utf8.
> 
> If mysql_enable_utf8 is set, most strings with SvUTF8 flag unset want to 
> be run through bytes_to_utf8.  I say most, as if you're populating a 
> BLOB, you don't want to upgrade your string to UTF-8, which DBD::mysql 
> probably can't determine.  Still, behaviour which doesn't depend on an 
> obscure internal flag would be a step forward!
> 
> This report may be a more general bug from the one in RT#25590.  I don't 
> think it's the same problem as RT#53130, which is concerned with data 
> being read.
> 
> Kind regards,
> Dave Lambley

Hi Dave! I belive I fix this UTF-8 problem in DBD::mysql. I reused your (slightly modified) test to verify that UTF-8 support is in DBD::mysql fixed. Changes are in my pull request at https://github.com/perl5-dbi/DBD-mysql/pull/67 Can you test & verify it?

From [email protected] on 2017-07-01 09:16:24:

Reopening, fix was reverted in 4.043.