python-ftfy icon indicating copy to clipboard operation
python-ftfy copied to clipboard

İ and ī not detected as mojibake

Open robstradling opened this issue 3 years ago • 0 comments

Hi @rspeer. Many thanks for creating and maintaining FTFY! We're using it at Sectigo to help prevent mojibake from finding its way into string fields in the digital certificates that we issue. We've noticed a couple of mojibake sequences that FTFY doesn't currently detect and fix:

Desired behaviour:

$ echo "İstanbul" | iconv -t WINDOWS-1252
İstanbul
$ echo "Rīga" | iconv -t WINDOWS-1252
Rīga

Current FTFY behaviour:

$ echo "İstanbul" | ftfy
İstanbul
$ echo "Rīga" | ftfy
Rīga

Would it be possible to make FTFY handle these cases?

robstradling avatar May 06 '22 22:05 robstradling