thunderbird-android icon indicating copy to clipboard operation
thunderbird-android copied to clipboard

Cyrillic folder names are displayed incorrectly

Open ke46138 opened this issue 7 months ago • 9 comments

Checklist

  • [x] I have used the search function to see if someone else has already submitted the same bug report.
  • [x] I will describe the problem with as much detail as possible.

App

K-9 Mail

App version

Manually builded from branch main from 23.05.2025

Where did you get the app from?

Other

Android version

Android 14 HIOS (stock rom)

Device model

Tecno Spark 30 (KL6)

Steps to reproduce

  1. Open app
  2. Create account
  3. Open folder list (left menu)
  4. See "?" symbols instead of Cyrillic characters in folder names

Expected behavior

Correct folder names, like this:

Image

Actual behavior

Signs "?" instead of Cyrillic characters in folder names.

Image

Logs

k9mail-logs.txt

ke46138 avatar May 23 '25 19:05 ke46138

Could you try to grab the logs at the time the login process occurs or let us know if your email server supports SMTPUTF8 ? What email provider are you using?

Does this work on Thunderbird for Android 10 and fail on main?

kewisch avatar May 26 '25 19:05 kewisch

your email server supports SMTPUTF8?

Yes. If you're interested, here's the EHLO test command output:

250-smtp.mail.ru
250-SIZE 73400320
250-8BITMIME
250-DSN
250-SMTPUTF8
250 AUTH PLAIN LOGIN XOAUTH2

Could you try to grab the logs at the time the login process occurs

logcat.txt

What email provider are you using?

mail.ru

Does this work on Thunderbird for Android 10 and fail on main?

K9-mail and Thunderbird 10.0 works well. Thunderbird from branch main have this bug.

This bug also appears on other phones with different versions of Android. I tested: Huawei Y5 Lite 2018 (Android 8), Alcatel Onetouch Pop 3 (5) (Android 5.1).

ke46138 avatar May 26 '25 21:05 ke46138

Ah I was thinking this might be a regression from #9171 but it looks like we haven't merged that yet. Maybe you can try to narrow it down a bit? We have a beta build, and there are daily builds on https://ftp.mozilla.org/pub/thunderbird-mobile/android/nightly/2025/

I'll see if we can take a look at this soon.

kewisch avatar May 27 '25 07:05 kewisch

Maybe you can try to narrow it down a bit?

In daily builds from May 12 there is this bug. And if you install a build that does not have this bug, log in to your account, install (upgrade an existing) a build that has this bug, the bug does not appear.

ke46138 avatar May 27 '25 09:05 ke46138

#9171 wouldn't be related to this.

I suspect that it's connected to something the mail.ru IMAP server does. That's a freemail service, right? Creating an account seems a little difficult for someone who doesn't know Russian. @ke46138 can you help me get an account there?

arnt avatar May 27 '25 13:05 arnt

can you help me get an account there?

Send me an email to [email protected] and I will send you the login and password.

I suspect that it's connected to something the mail.ru IMAP server does.

I doubt. Gmail has the same problem.

That's a freemail service, right?

Yes

ke46138 avatar May 27 '25 15:05 ke46138

I can reproduce this on Gmail by creating a folder on Thunderbird Desktop named "Исходящие" and then going to Thunderbird for Android.

kewisch avatar May 28 '25 10:05 kewisch

Oh, cool, then it should be easy to reproduce. I'm charging a test device now and can start working on this once it's charged.

arnt avatar May 28 '25 10:05 arnt

Have reproduced the problem and have a failing unit test. More tomorrow.

arnt avatar May 28 '25 14:05 arnt

I pushed code that works for me. I suspect that it may break in a case that I would like to test automatically, but don't know how to write:

  1. Mock a plain IMAP connection and create a folder with a unicode name based on it, as well as a message in INBOX.
  2. Close the connection.
  3. Issue a copy for the message in INBOX to the folder with a cyrillic name, in the same way that the user interface issues a move.
  4. Assert that the queue of pending commands contains a copy.
  5. Mock a new IMAP connection with UTF8=ACCEPT.
  6. Assert that the copy operation uses the correct folder name.

This isn't exactly a unit test, and writing it requires fairly wide-ranging knowledge of Thunderbird. I think the test is needed, but if I am to write it I need a hint or two.

arnt avatar Jul 01 '25 14:07 arnt

Oh, and the point of the test is to check the behaviour a user sees in the first minutes after Thunderbird and the IMAP server have both been extended.

arnt avatar Jul 01 '25 14:07 arnt

This bug is not about Cyrillic script, but also about Japanese. Used both, both displayed incorrectly

WHIFAN avatar Jul 21 '25 13:07 WHIFAN

I believe it applies to almost all languages and scripts, Cyrillic is just the script for which the issue was discovered. The fix should also resolve every script that is written horizontally and has font support.

arnt avatar Jul 21 '25 13:07 arnt

Is anyone hitting this on 11.0?

coreycb avatar Jul 21 '25 14:07 coreycb

You should never hit this on 11.0, it was introduced as a side effect of code I added after that.

The short version is: adding support for all of unicode in email addresses also added support for the rest of RFC 6855/9755, and that broke ⓐ support for unicode folder names (because of a cleanup in the RFCs) and ⓑ a unit test (for complicated reasons). #9241 fixes the former, and when I've time to learn mockito-kotlin I'll fix the unit test as well, and then we'll hopefully all be happy.

arnt avatar Jul 21 '25 20:07 arnt

https://github.com/thunderbird/thunderbird-android/issues/9480

pagebug avatar Jul 22 '25 17:07 pagebug