Unable to fully show Chinese characters [MacOS 11][M1][Native Build 5.18]

[qinyongqiang]

1. Operating system Mac OS 11.4, M1
2. Transmission Deamon 3.0.0
3. Transmission Remote GUI 5.18.0, native built version

Inspired by this #1335, I successfully built the native M1 version (also, the binary on that issue also works for me). Almost everything works fine except display of Chinese characters.

For example, if the file name is ABCD, it shows like A__C__ or A?C?D

What would be wrong, or how can I have a fix in building settings or source code?

[YaxinCheng]

I'm having the same problem. Actually, it's not just for Chinese, Japanese (Kana & Kanji) also has such issue. Have you had any progress on this?

[elisimpson]

I have a similar issue where a tilde (~) in the filename of the subfiles for a torrent is changed automatically to an underscore in the interface. If you try to rename any of the subfiles to something with a tilde character it throws an "Invalid Argument" error. After I get this error I can no longer rename the subfile at all and to delete the torrent.

[shermanikk]

Well I'm not sure quite what fixed it, but after compiling with fpc 3.3.1 and Lazarus 2.3.0 the issue seems to have resolved itself for me. I'm running 12.1 on an M1 MacBook Air and the name for this torrent is now showing up correctly:

If you want to try my build, see if it fixes things for you it's up for download here

[qinyongqiang]

Well I'm not sure quite what fixed it, but after compiling with fpc 3.3.1 and Lazarus 2.3.0 the issue seems to have resolved itself for me. I'm running 12.1 on an M1 MacBook Air and the name for this torrent is now showing up correctly:

If you want to try my build, see if it fixes things for you it's up for download here

Yea...I just tried your built. It works seamlessly! Thanks for sharing!

[YaxinCheng]

Well I'm not sure quite what fixed it, but after compiling with fpc 3.3.1 and Lazarus 2.3.0 the issue seems to have resolved itself for me. I'm running 12.1 on an M1 MacBook Air and the name for this torrent is now showing up correctly:

If you want to try my build, see if it fixes things for you it's up for download here

Can confirm this is fixed. Thanks a lot

[lighterowl]

tl;dr It's basically this issue which was fixed in FPC 3.3.1.

This also occurs on Linux with the newest Lazarus/FPC versions available at the time of writing (2.2.4/3.2.2) in Arch Linux.

I suspect - but have no hard evidence - that this might be related to the fact that Transmission (up to and including 3.0) uses JSON's \uXXXX encoding to represent non-ASCII characters - this is an example reply :

{
  "arguments": {
    "torrents": [
      {
        "name": "\u304d\u306e\u3053\u5e1d\u56fd"
      }
    ]
  },
  "result": "success"
}

Putting this through jq produces a valid representation which uses UTF-8, which proves that the source data is OK :

$ jq . <<< '{"arguments":{"torrents":[{"name":"\u304d\u306e\u3053\u5e1d\u56fd"}]},"result":"success"}'
{
  "arguments": {
    "torrents": [
      {
        "name": "きのこ帝国"
      }
    ]
  },
  "result": "success"
}

$ jq -c . <<< '{"arguments":{"torrents":[{"name":"\u304d\u306e\u3053\u5e1d\u56fd"}]},"result":"success"}' | xxd
00000000: 7b22 6172 6775 6d65 6e74 7322 3a7b 2274  {"arguments":{"t
00000010: 6f72 7265 6e74 7322 3a5b 7b22 6e61 6d65  orrents":[{"name
00000020: 223a 22e3 818d e381 aee3 8193 e5b8 9de5  ":".............
00000030: 9bbd 227d 5d7d 2c22 7265 7375 6c74 223a  .."}]},"result":
00000040: 2273 7563 6365 7373 227d 0a              "success"}.

However, TransGUI renders this as :

Interestingly, it seems like only every second character is affected while the rest is fine.

[lighterowl]

Compiling with fpc 3.2.3 (current top of fixes_3_2) fixes this as well.

[Gfurst]

I'm still getting this as reported by @xavery above.

Wasn't the fix sent upstream? Or do we need to patch something ourselves?

[lighterowl]

The fix is in FPC 3.2.3 which is the current development version of FPC. It is not available in any stable versions yet.

The only way to get this fixed for now is to compile with FPC 3.2.3.

[15cm]

I also confirmed the fix on NixOS by using 3.2.3 fpc on top of the fixes_3_2 branch: https://github.com/15cm/nixfiles/commit/caf06cbb35bf7466b2d521fb55071318a854fa35