SteamOS icon indicating copy to clipboard operation
SteamOS copied to clipboard

Some Games Require setting LC_ALL Variable for correct Language. Not possible due to stripped glibc package.

Open NightHammer1000 opened this issue 2 years ago • 23 comments

Some Games Require that you pass LANG= %command% or LC_ALL= %command%

To launch the Game in the Correct Language. Some Games Default to English because thats the set System Language.

Doing this is not possible because the installed glibc was stripped of all its Language Files. Which is, in my opinion, a big no-go.

Attempting to enable a locale in /etc/locale.gen and running locale-gen results in:

Generating locales...
  de_DE.UTF-8...failed to set locale!
[error] cannot open locale definition file `de_DE': No such file or directory

This can only be fixed by reinstalling glibc via pacman.

The ability to set locales should be shipped in the stock Image. There is no reason why it shouldn't.

NightHammer1000 avatar Jun 06 '22 10:06 NightHammer1000

Hey, are you talking about Steam games or about games that you installed manually on the desktop? Do you have some examples?

bertogg avatar Jun 07 '22 16:06 bertogg

Patrician IV Is one of the Games not setting the Language correctly without the possibility to set locals.

NightHammer1000 avatar Jun 07 '22 16:06 NightHammer1000

What version of this game are you running? The one from Steam fails with Error in 'App::Initialize'.

Edit: never mind, it worked on the third attempt

bertogg avatar Jun 07 '22 16:06 bertogg

From the Steam UI it certainly does ignore the language setting, how are you launching it to make it work in German?

bertogg avatar Jun 07 '22 17:06 bertogg

LC_ALL=de_DE.UTF-8 %command% does the trick for me. You need to delete the Wine Prefix tho, because once the config file is created there is no way to change it again without editing the Config File.

NightHammer1000 avatar Jun 07 '22 18:06 NightHammer1000

Another Example is a Non-Steam one. Bioshock Remastered from the Epic Store Launched through Heroic. This Game also requires that you pass LC_ALL=de_DE.UTF-8 to it.

NightHammer1000 avatar Jun 07 '22 18:06 NightHammer1000

I note that even on systems that have a glibc install supporting all locales, these title still require manual workarounds to enable specific languages.

TTimo avatar Jun 07 '22 19:06 TTimo

In the case of Patrician IV setting the language from English to German downloads the German version of the game (>2GB) with all voices in German, but all messages on the screen, menus, etc, are still in English.

bertogg avatar Jun 08 '22 11:06 bertogg

The language of the messages is defined in c:/users/steamuser/Application Data/Kalypso Media/Patrician 4/options.cfg

If you install the German version of the game and never run the English version before then everything works as expected.

The problem happens if you run the English version first and then switch to the German version. I would say that's a bug, and it can be worked around by removing (or editing) options.cfg after switching versions.

None of this requires changes to the available glibc locales as far as I can tell.

bertogg avatar Jun 09 '22 16:06 bertogg

Still. Is there an explicit Reason why all the Language files should be stripped? Why not give the users the Possibility to run the Desktop and Non-Steam Games in their own language? The fact that the Language Files are stripped out is something I don't get. Even the smallest Live-CD Distros like GParted and DamnSmallLinux are supplying those files.

NightHammer1000 avatar Jun 09 '22 18:06 NightHammer1000

Why not give the users the Possibility to run the Desktop and Non-Steam Games in their own language?

We believe this is possible already, and does not require the extra libc locale files. @bertogg has concluded that Pratician IV failing to switch to German is a game bug. Those typically need to be taken up with the game developer.

We aggressively reduce the size of the OS partition by removing locales, man pages, etc. This is done to leave as much space as possible for game content. We understand this is not ideal for a desktop use case.

Some of these decisions on what to include/exclude may change in the future, but in this particular case we do not see this being justified yet.

TTimo avatar Jun 09 '22 19:06 TTimo

The Game was one example fixable with other means. Games like Bioshock Remasterd from the Epic Store through Herioc is one example that is not fixable otherwise.

But that issue goes even further than games. I know of at least 2 German Steam Deck users that sold their devices because Desktop Mode was unusable for them because their English is not good enough. Not every single person knows english. I do believe this is a Point you guys should really think about when selling Internationally. If you do not ship them, at least provide a Script that can pull those files from a Server.

NightHammer1000 avatar Jun 10 '22 08:06 NightHammer1000

I think there are two things here:

  • Whether having all locales generated is necessary to run a particular game in a particular language. That I would be interested to know in order to see what's actually happening under the hood and how we can solve it. In the Patrician IV case that doesn't seem to be the problem and in principle I would expect that you can change any game's language without having to change the locale.

  • Whether the OS should support running the desktop with any arbitrary locale and language. Here there are more considerations, one of them (not the only one) is that we would need all translation files for all packages, in my computer that's more than half a GB worth of files in the root filesystem.

bertogg avatar Jun 10 '22 16:06 bertogg

a large portion of visual novels depend on japanese locale to be able to run. not having japanese locale in steamos is cutting out a large portion of an entire genre of game from running. these are non-steam games, but all the same it would be really great to be able to run them. please consider adding japanese locale for visual novel support.

there are a number of examples of people trying to attempt this and running into issues that require them to unlock steamos and hoping for the best.

https://old.reddit.com/r/SteamDeck/comments/u77i9m/steam_deck_japanese_visual_novels_how_change/ https://old.reddit.com/r/SteamDeck/comments/wc14gg/enable_japanese_locale_on_steam_deck/ https://old.reddit.com/r/SteamDeck/comments/v5jcbo/guide_to_changing_locale_settings_for_japanese/ https://old.reddit.com/r/SteamDeck/comments/v8375c/how_does_proton_handle_locale_emulation/

a guide on youtube someone made about unlocking the steam deck to be able to play vns on the steam deck: https://www.youtube.com/watch?v=KtiA6GaFIzM&t=3s

xrishox avatar Aug 24 '22 18:08 xrishox

Non-steam game or not, running these in a sniper SLR would be the better way forward in order to avoid switching the root OS to read/write. We are working on improving locale support but that has a lot of ramifications, no ETA.

TTimo avatar Aug 24 '22 19:08 TTimo

The Japanese locale can be enabled on the Deck without unlocking SteamOS. If the game's files only have ASCII characters: using HOST_LC_ALL=ja_JP.UTF-8 %command% as launch options does the trick of setting Proton's codepage (the locale doesn't have to actually exist on the system in this case).

For games that both require the Japanese locale in Proton + have Japanese filenames: ja_JP.UTF-8 locale files can be created on a full Linux install then copied to the Deck's user directory + using a custom launch script to overwrite LOCPATH after the Steam Runtime resets it.

(And for games that use non-English filenames but don't require changing Proton's locale: just LANG=en_US.utf8 %command% is enough. utf8 instead of UTF-8. SteamOS bug? It also affects apps running in desktop mode.)

Non-root solution at https://gist.github.com/cfillion/4394c3b8cd051fb45721187053e92296.

cfillion avatar Aug 25 '22 01:08 cfillion

:man_facepalming: When I look at this conversation in 2022 and the age of the internet, I am at the point where I feel it is sickening that we even have to have this kind of conversation, just because some narrow minded English only speaking decision makers are a) either totally oblivious or ignorant to the fact that the rest of the world does not speak English and b) customers from around the world are paying for a product. :disappointed: The sooner you get into your head that proper language support IS a huge issue when selling internationally the better for everybody.

  • Whether the OS should support running the desktop with any arbitrary locale and language. Here there are more considerations, one of them (not the only one) is that we would need all translation files for all packages, in my computer that's more than half a GB worth of files in the root filesystem.

This is not true. And, you know it.

We aggressively reduce the size of the OS partition by removing locales, man pages, etc. This is done to leave as much space as possible for game content. We understand this is not ideal for a desktop use case.

In fact, it is not ideal in the primary nor any other use case, regardless if we look at the Steam Deck or any other SteamOS device. While minimizing system storage space requirements is a justified and reasonable engineering goal (especially on resource constrained devices), it is far from reasonable to solve this optimization problem by blatantly dropping all locales. Most users usually want or need support for no more than a handful of locales. You can still optimize your storage space utilization by leaving out all the other locales that users do not need. People will happily trade in a little bit of usable storage space on the device for being able to read and understand what they are doing. Linux distros including SteamOS can be configured to a set of supported locales (see /etc/locale.gen and locale-gen). Hence, there is a mechanism to download or install only needed locales.

Sure, you can tinker in yourself all the locales you might need but this is not how this is supposed to work 2022 and on a device that is set out to consolidate the world of gaming. If Valve really wants to win over significant portions of Microsoft’s gaming and OS market share permanently you better quickly start taking full locale support seriously because it is not a side issue for the non‑English speaking world.

Some of these decisions on what to include/exclude may change in the future, but in this particular case we do not see this being justified yet.

On the contrary, it has been justified since day one of release. The pure and simple justification is namely the fact that you are shipping and selling a product to a global audience. So, I do really hope that these “decisions” are going to happen sooner than later.

We are working on improving locale support but that has a lot of ramifications, no ETA.

Good, but please make it a true priority, not just some time in the future blah blah blah… Proper localization support is a bigger deal for your customers than you may realize.

JakeSmarter avatar Sep 05 '22 11:09 JakeSmarter

I'm not from Valve and I don't decide what locales to support, I was just trying to understand the problem and give some technical context, but anyway:

I feel it is sickening that we even have to have this kind of conversation, just because some narrow minded English only speaking decision makers are a) either totally oblivious or ignorant [...] This is not true. And, you know it.

This is unnecessary.

Linux distros including SteamOS can be configured to a set of supported locales (see /etc/locale.gen and locale-gen). Hence, there is a mechanism to download or install only needed locales.

There are two things: compiling the locale definitions and actually having the translations installed. The translations are what takes most of the disk space and they come with each individual package. Also, note that the root filesystem is read-only.

bertogg avatar Sep 05 '22 14:09 bertogg

We can argue all day here. The fact is that SteamOS currently Gatekeeps the Desktop Mode and certain Games to English Speakers only, only to save a few Megabytes really not worth saving if it means that half of the Device is not usable at all to non Native English speakers. Unnecessary in my eyes. I don't understand why this is a point of discussion at all. It should be a given to support all Translations if you sell worldwide. ESPECIALLY if all the Translations already exist.

NightHammer1000 avatar Sep 05 '22 16:09 NightHammer1000

There are two things: compiling the locale definitions and actually having the translations installed. The translations are what takes most of the disk space and they come with each individual package

Certainly, I think that when talking about locales we all mean or agree that locale definitions without translations are worthless. So, there is no need to make a more granular distinction in this context. In fact, one could argue that there is even more to full locale support, e.g. fonts. However, this has been conveniently solved either by Valve or KDE by using Noto (I do not know for sure who made that decision) by default in Desktop Mode.

Anyhow, it is true that many package maintainers unfortunately do not provide separate langpacks but rather consolidate translations into one package which does make things more complicated in this context. However, with almost every sane package management system, including Pacman and apt, you can configure pre‐ and post‑install events, exclude certain directories from writing during install, or just exclude some files and directories in the package from processing beforehand. In Pacman these install events are called “Hooks”. Hence, you could delete translations for unsupported locales on the target machine on post‑install or exclude directory names by unsupported locale identifiers from writing. So, there are ways and solutions to deal with the problem. Valve just needs to take full locale support seriously and implement a solution which ought to be not that difficult despite some odds.

Also, note that the root filesystem is read-only.

Right, but this is just a system design and configuration decision for safety and security reasons and has nothing to do with locale support per se or on its own. Besides, any OTA update or package management write operation is either elevated to root permissions or the write‑lock on the root filesystem is temporarily lifted already. So, this is not an issue.

Besides,

I feel it is sickening that we even have to have this kind of conversation, just because some narrow minded English only speaking decision makers are a) either totally oblivious or ignorant […]

This is unnecessary.

Unfortunately, IT IS necessary because extremely well paid decision makers do not realize the size of the mess they have created by making poor decisions.

JakeSmarter avatar Sep 07 '22 07:09 JakeSmarter

I am currently thinking about forking HoloISO and modifying it from its hardware universal approach back to a Steam Deck-specific Image. Just to give the users the Option to something Valve is unwilling to give. Their Native Language and an open file system.

Like I said. I know a few people, mostly german, who were really excited about the prospect of a Portable PC with full Desktop access but canceled their order once they got to know that Desktop was English only.

Due to the changes to the kernel not being upstreamed, we currently have only two real options. SteamOS with very Lacking International functionality, and Windows, which Drivers lack important functionality like DXVA2 while also having massive Installing Problems. (Bluescreen with no Boot after APU Driver install)

Trying to run a different Distro on the Deck will give you a headache.

So, our best chance currently is to build the solution we want ourselves from the Image we already have. Because I don't see something changing here anytime soon, to be honest.

NightHammer1000 avatar Sep 07 '22 20:09 NightHammer1000

they have already acknowledged the problem and said they plan to address it. i don't know why you are acting like this is not the case. it will get done when it gets done. this is not the place to have a passive aggressive and disrespectful back and forth about it. it won't get it fixed any faster.

in the mean time, unlike any other console that is currently produced, you can unlock root and fix it yourself until it gets pushed out officially if it is a serious problem for you.

xrishox avatar Sep 07 '22 21:09 xrishox

We aggressively reduce the size of the OS partition by removing locales, man pages, etc.

@TTimo FYI: There are better ways to do this, see Installing only content in required languages. The same applies to man pages, etc.

By the way, the root locale is not en_US but C. Thus, the default and fallback locale should be C.UTF-8.

JakeSmarter avatar Sep 17 '22 12:09 JakeSmarter

Steam Deck just launced in Asia and guess what? The desktop mode is STILL in English only. Also, it looks like some games (on Unity?) cannot display japanese language, for example RimWorld and Valheim, due to missing fonts I suppose. They both have "Verified" mark btw.

nisehime avatar Dec 18 '22 06:12 nisehime

Seems like user Experience still isn't a huge priority

NightHammer1000 avatar Dec 18 '22 10:12 NightHammer1000

locale missing introduced a lot troubles for me and spends me almost a week to address them.

There is multiple solutions imo:

  1. Valve delete built-in desktop wallpapers to save disk space for pre-installed locale files
  2. Valve add new SteamOS-offload mount to make locale-gen works
  3. User use overlayfs to make locale-gen works (e.g. rwfus)

StoneMoe avatar Dec 19 '22 04:12 StoneMoe

Another Example for Software that does not work in other Languages is the Emulator PPSSPP in Flatpak form because Flatpak does not download the Locals if it's not set. So, you are unable to set the Language of the Emulated PSP, and it always defaults to English in all Games you Launch with it.

NightHammer1000 avatar Dec 24 '22 16:12 NightHammer1000

Another Example for Software that does not work in other Languages is the Emulator PPSSPP in Flatpak form because Flatpak does not download the Locals if it's not set. So, you are unable to set the Language of the Emulated PSP, and it always defaults to English in all Games you Launch with it.

This can be workaround by:

flatpak config --set language "en;jp;fr" # change language code here, add --user if your flatpaked app is installed as user app
flatpak update  # flatpak will download locales you set

StoneMoe avatar Jan 05 '23 06:01 StoneMoe

Thanks. Man. Steam Deck outside the English Language really turns into a workaround machine.

NightHammer1000 avatar Jan 05 '23 08:01 NightHammer1000

flatpak config --set language "en;jp;fr" # change language code here, add --user if your flatpaked app is installed as user app flatpak update # flatpak will download locales you set

does this work even without steamos itself having japanese locale enabled?

xrishox avatar Jan 07 '23 19:01 xrishox

does this work even without steamos itself having japanese locale enabled?

Locale files can be installed anywhere, they don't need to be system-wide.

cfillion avatar Jan 07 '23 20:01 cfillion

just thought i might post this since it worked well for me, i just set up a debian distrobox and installed all of the locale/wine/lutris/whatever stuff in there and so far its been working perfectly for the couple of locale dependent games i've tried. from there you could probably find a way to add them as steam games if you got creative with it. it's nice because since it's all in a distrobox its all sequestered from the rest of my system, it didn't require that i unlock my os, and it should just persist as you would expect through updates.

xrishox avatar Jan 08 '23 03:01 xrishox

flatpak config --set language "en;jp;fr" # change language code here, add --user if your flatpaked app is installed as user app flatpak update # flatpak will download locales you set

does this work even without steamos itself having japanese locale enabled?

yes, locale files for flatpak apps is managed by flatpak, so it's OS independent

StoneMoe avatar Jan 08 '23 05:01 StoneMoe