python-ftfy icon indicating copy to clipboard operation
python-ftfy copied to clipboard

Mojibake problem: ¿

Open MicroJackson opened this issue 4 years ago • 1 comments

Hi, Think this one is not possible, but here it is:

U+00BF  ¿       [Po] INVERTED QUESTION MARK
SHOULD BE:
U+2011  ‑       [Pd] NON-BREAKING HYPHEN

But then in other texts:

U+00BF  ¿       [Po] INVERTED QUESTION MARK
SHOULD BE:
U+0027  '       [Po] APOSTROPHE
Or maybe:
U+2019  ’       [Pf] RIGHT SINGLE QUOTATION MARK

Example:

of UC¿TM zo¿n beetje alle camera¿s

The second one is a huge issue, because almost all plural forms of a thing uses the quotation+s syntax in Dutch.

Is it possible to fix this?

MicroJackson avatar Mar 12 '20 11:03 MicroJackson

There's probably too little information to fix this string, but I'm leaving this one open so I can try to figure out what combination of encodings made this happen.

rspeer avatar Mar 12 '20 14:03 rspeer