Errors with UTF-8 filenames with UTF-8 flag on [rt.cpan.org #67431]

[atoomic]

Migrated from rt.cpan.org#67431 (status was 'new')

Requestors:

• [email protected]

From [email protected] on 2011-04-13 13:48:22:

When you use UTF-8 template file names with non-ascii characters, TT  
sometimes fails with an error "file error - 1: not found".
The reason is that Perl's %INC hash always clears the UTF-8 flag on its  
keys, so when Template::Provider::_load_compiled() does delete $INC{$file}  
with $file being a UTF-8 string with flag ON, the entry is not really  
removed from %INC, and "require()" returns 1.
How to fix: add the line
  Encode::_utf8_off($file);
before
  delete $INC{$file};
My system is Debian sid with Perl 5.10.1.

How to reproduce:

#!/usr/bin/perl

use Template;
use Encode;

my $fn = 'test-тест.tt';
{
     open FD, '>', $fn;
     print FD "hello world\n";
     close FD;
}
my $template = new Template({COMPILE_DIR => './', COMPILE_EXT => 'c'});
Encode::_utf8_off($fn);
$template->process($fn, {}) || die $template->error;
$template = new Template({COMPILE_DIR => './', COMPILE_EXT => 'c'});
$template->process($fn, {}) || die $template->error;
Encode::_utf8_on($fn);
$template->process($fn, {}) || die $template->error;

__END__

Here, the last process() gives the error "file error - 1: not found".

-- 
With best regards,
   VitaliyFilippov

[FGasper]

Encode::_utf8_on effectively does a UTF-8 decode, which changes the Perl string’s logical value. So of course the last process() will fail: you’re giving it a different string.

Notably, if you utf8::upgrade($fn) instead of _utf8_on($fn) it works; thus, there’s no string-storage abstraction leak bug here.

The example is invalid. Suggest closing.