[Bug]: not all languages recognised with mixed locale and thus not used for Locale extensions

[BarbzYHOOL]

Checklist

• [X] I agree to follow the Code of Conduct that this project adheres to.
• [X] I have searched the issue tracker for a bug that matches the one I want to file, without success.
• [X] If this is an issue with a particular app, I have tried filing it in the appropriate issue tracker for the app (e.g. under https://github.com/flathub/) and determined that it is an issue with Flatpak itself.
• [X] This issue is not a report of a security vulnerability (see here if you need to report a security issue).

Flatpak version

Flatpak 1.14.4

What Linux distribution are you using?

Pop!_OS

Linux distribution version

pop os 22.04

What architecture are you using?

x86_64

How to reproduce

No response

Expected Behavior

I am using

flatpak --version: Flatpak 1.14.4, and
dpkg -s libglib2.0-bin | grep '^Version:' Version: 2.72.4-0ubuntu2.2

on Pop_OS! 22.04 with a mixed locale:

LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_CTYPE="en_US.UTF-8" LC_NUMERIC=fr_FR.UTF-8 LC_TIME=fr_FR.UTF-8 LC_COLLATE="en_US.UTF-8" LC_MONETARY=fr_FR.UTF-8 LC_MESSAGES="en_US.UTF-8" LC_PAPER=fr_FR.UTF-8 LC_NAME=fr_FR.UTF-8 LC_ADDRESS=fr_FR.UTF-8 LC_TELEPHONE=fr_FR.UTF-8 LC_MEASUREMENT=fr_FR.UTF-8 LC_IDENTIFICATION=fr_FR.UTF-8 LC_ALL=

and localectl list-locales

C.UTF-8
en_AG.UTF-8
en_AU.UTF-8
en_BW.UTF-8
en_CA.UTF-8
en_DK.UTF-8
en_GB.UTF-8
en_HK.UTF-8
en_IE.UTF-8
en_IL.UTF-8
en_IN.UTF-8
en_NG.UTF-8
en_NZ.UTF-8
en_PH.UTF-8
en_SG.UTF-8
en_US.UTF-8
en_ZA.UTF-8
en_ZM.UTF-8
en_ZW.UTF-8
es_AR.UTF-8
es_BO.UTF-8
es_CL.UTF-8
es_CO.UTF-8
es_CR.UTF-8
es_CU.UTF-8
es_DO.UTF-8
es_EC.UTF-8
es_ES.UTF-8
es_GT.UTF-8
es_HN.UTF-8
es_MX.UTF-8
es_NI.UTF-8
es_PA.UTF-8
es_PE.UTF-8
es_PR.UTF-8
es_PY.UTF-8
es_SV.UTF-8
es_US.UTF-8
es_UY.UTF-8
es_VE.UTF-8
fr_BE.UTF-8
fr_CA.UTF-8
fr_CH.UTF-8
fr_FR.UTF-8
fr_LU.UTF-8

flatpak config and flatpak --user config don't report the fr:

languages: *unset* (default: en)
extra-languages: *unset*

It should already include https://github.com/flatpak/flatpak/pull/4411, yet French is not selected as a locale for me.

This led to https://github.com/hugolabe/Wike/issues/136 where the WebKit view does not work with its process crashing, similarly to https://github.com/flatpak/flatpak/issues/3712:

(process:2): Gtk-WARNING **: 01:23:09.738: Locale not supported by C library.
	Using the fallback 'C' locale.
flatpak-spawn: Invalid byte sequence in conversion input
Try "flatpak-spawn --help" for more information.

Actual Behavior

(process:2): Gtk-WARNING **: 01:23:09.738: Locale not supported by C library.
	Using the fallback 'C' locale.
flatpak-spawn: Invalid byte sequence in conversion input
Try "flatpak-spawn --help" for more information.

Additional Information

https://github.com/hugolabe/Wike/issues/136

[steffenskov]

I too have this issue, running en_DK.UTF-8 as my default locale, yet flatpak reports *unset* for me as well, and I get the dreaded Locale not support by C library :-/ I'm on OpenSuse Tumbleweed using Gnome

[steffenskov]

Update, from reading the other report on hugolabe I found this (somewhat simple) workaround:

sudo flatpak override --env=LC_ALL=en_DK.UTF-8 io.gitlab.news_flash.NewsFlash

Feel free to replace en_DK with your language of choice and io.gitlab.news_flash.NewsFlash with the flatpak in question. This is just forcing that locale on everything in the flatpak, and at least for me that's sufficient to solve matters :)

[BarbzYHOOL]

yes but it's hacky

[steffenskov]

yes but it's hacky

Complete agree, it's definitely not a complete solution, more of a workaround for the time being until the actual issue is fixed :-)

[darkdragon-001]

As an additional data point, I am having the same problem but use the following language settings:

LANG=en_US.UTF-8
LANGUAGE=en
LC_ADDRESS=de_DE.UTF-8
LC_IDENTIFICATION=de_DE.UTF-8
LC_MEASUREMENT=de_DE.UTF-8
LC_MONETARY=de_DE.UTF-8
LC_NAME=de_DE.UTF-8
LC_NUMERIC=de_DE.UTF-8
LC_PAPER=de_DE.UTF-8
LC_TELEPHONE=de_DE.UTF-8
LC_TIME=de_DE.UTF-8

As I install most Flatpak apps as user, I used flatpak --user (without sudo) to overwrite both LANG and LC_ALL to en_US.UTF-8 to work around the problem.

I am on Ubuntu 23.10 with GNOME.

[jntdofe]

Same bug just happened with me in Fedora Silverblue 39, Flatpak 1.15.6 causing Epiphany 46 to crash WebKit (https://gitlab.gnome.org/GNOME/epiphany/-/issues/2299)

A very easy way to reproduce it in GNOME is to change GNOME Settings "formats" in "Region & Language" to a locale that doesn't match the language that you have, after a logout login Flatpak doesn't recognizes that it has to download the new added locale (flatpak update does nothing) and the mismatch cause some apps to crash.

A workaround for this issue in GNOME is to change the language of the system to match the locale you desire, logout and login, update Flatpak with flatpak update change the language back to the original, logout and login. This will make the apps work again with mixed locales.

Maybe Flatpak should detect the missing locale and ask for an update but there is probably a wiser solution that I'm not seeing to just make it not crash.

[PVermeer]

@jntdofe

The current workaround is to let flatpak know that your using mixed languages:

For me:

flatpak config --set languages "en;nl"
flatpak update

[chrisawi]

A very easy way to reproduce it in GNOME is to change GNOME Settings "formats" in "Region & Language" to a locale that doesn't match the language that you have

I was sure this used to work, at least for system locales (via localed). It looks like it regressed recently in https://github.com/flatpak/flatpak/pull/5018. Flatpak now exclusively uses AccountsService's GetUsersLanguages method when available. Ideally, that method should include all languages mentioned in the system locale instead of just LC_MESSAGES, but I can't see any real downside to adding back the localed query in the meantime.

Harder to fix is the multi-user case where the locale is set via AccountsService rather than localed. In GNOME, currently only the primary language is set there; the Formats locale is set in a GSettings key, so flatpak can't know about it without GNOME-specific code. In GNOME/gnome-control-center#1969 (comment), @hadess suggested that Formats should select between languages that have been set in AccountsService, which would nicely solve this issue for flatpak.

The user installation does work correctly because it honors the LC_* variables set in the environment.

---

Separately, it would nice if glibc could fall back to C.UTF-8 instead of C, now that it exists upstream. If that can't happen, maybe flatpak should do that itself for locales it knows are unavailable (while complaining loudly).

[jantari]

I am experiencing this issue with the Thunderbird Flatpak.

I'm using Ubuntu 24.04 with GNOME, language is "English (United States)" and formats are "Deutschland" (Germany).

~ 
❯ env | egrep "LC|LANG" | sort
LANG=en_US.UTF-8
LC_ADDRESS=de_DE.UTF-8
LC_IDENTIFICATION=de_DE.UTF-8
LC_MEASUREMENT=de_DE.UTF-8
LC_MONETARY=de_DE.UTF-8
LC_NAME=de_DE.UTF-8
LC_NUMERIC=de_DE.UTF-8
LC_PAPER=de_DE.UTF-8
LC_TELEPHONE=de_DE.UTF-8
LC_TIME=de_DE.UTF-8

However, in the Thunderbird Flatpak the dates of all e-Mails are in US timestamp format:

which is honestly unusable, but the Thunderbird settings only let me choose "English (United States)" - my real local regional formats (Germany/German) aren't available:

[chrisawi]

@jantari Is that set via localectl? If you have multiple user accounts, then GNOME will use AccountsService instead, and not in a way that lets flatpak get the secondary languages, as described above.

The flatpak system installation does not (and cannot) respect the values of LC_* in the user environment.

[jantari]

@chrisawi I think I just set this via the Ubuntu installer, pretty sure it asked for language and locale/format preference.

I can now see this choice in the GNOME Settings -> System -> Region & Language GUI.

I never used localectl before. localectl status shows:

~ 
❯ localectl status
System Locale: LANG=en_US.UTF-8
    VC Keymap: (unset)         
   X11 Layout: us
    X11 Model: pc105
  X11 Variant: intl

Is there a localectl command I can run to fix this (Thunderbird)?

EDIT: regarding multiple users, the only non-system users I have on this system (UID >= 1000) are jantari (uid 1000, my daily use account) and libvirt-qemu (uid 64055). So I'd say no, I don't really have multiple user accounts besides what is pre-configured on the system by Ubuntu.

[chrisawi]

Is there a localectl command I can run to fix this (Thunderbird)?

You can at minimum do localectl set-locale LC_TIME=de_DE.UTF-8 (you probably should set the rest as well)

That should cause de to appear as a default language in flatpak config, after which flatpak update will install the correct locale subsets.

Alternatively, you can manually add de to extra-languages via flatpak config (see the man page).

EDIT: regarding multiple users, the only non-system users I have on this system (UID >= 1000) are jantari (uid 1000, my daily use account) and libvirt-qemu (uid 64055). So I'd say no, I don't really have multiple user accounts besides what is pre-configured on the system by Ubuntu.

That's odd, GNOME definitely uses localed here on Fedora (I just checked). You could try switching the formats language in GNOME Settings to US and then back to Germany to see if that makes it show up in localectl. Ubuntu's installer may have done things differently.

[jantari]

You could try switching the formats language in GNOME Settings to US and then back to Germany to see if that makes it show up in localectl.

Well who would have thought that - this silly suggestion actually did it. Switching the "formats" to US-English, not even logging out, and immediately switching it back to German changed the output of localectl status:

~ 
❯ localectl status
System Locale: LANG=en_US.UTF-8
               LC_NUMERIC=de_DE.UTF-8
               LC_TIME=de_DE.UTF-8
               LC_MONETARY=de_DE.UTF-8
               LC_PAPER=de_DE.UTF-8
               LC_NAME=de_DE.UTF-8
               LC_ADDRESS=de_DE.UTF-8
               LC_TELEPHONE=de_DE.UTF-8
               LC_MEASUREMENT=de_DE.UTF-8
               LC_IDENTIFICATION=de_DE.UTF-8
    VC Keymap: (unset)                      
   X11 Layout: us
    X11 Model: pc105
  X11 Variant: intl

and flatpak config:

~ 
❯ flatpak config
languages: *unset* (default: de;en)
extra-languages: *unset*

One flatpak update later and Thunderbird now offers me "German (Germany)":

I guess this is Canonicals fault, but what a bad user experience. Then again, I doubt they care as they're doing anything in their power to push snaps anyway... but thank you a lot for your help! At least I can use Thunderbird now :)

[Viliansh]

I guess this is Canonicals fault, but what a bad user experience. Then again, I doubt they care as they're doing anything in their power to push snaps anyway... but thank you a lot for your help! At least I can use Thunderbird now :)

to be fair with them, i get the same issue with Fedora KDE, i didn't even knew i could use english with a different date format in thunderbird(i thought it was thunderbird fault idk) thank you for the report and chrisawi for the fix!