[5.2] Language fallback for special languages like 'de-CH', 'de-AT' for extensions providing the fallback language

[MacJoom]

Summary of Changes - thanks go to Allon @laoneo and Corinne @webuniverse for discussion and code

• Add an additional tag <fallbackTag> </fallbackTag> to langmetadata.xml (admin & frontend) e.g. <fallbackTag>de-DE</fallbackTag> for a fallback to de-DE so extensions will use this fallback language (seperate PR will be made for the langmetadata.xml files, after this PR passes...)

• process the tag in Language.php (load)

Testing Instructions

• A Joomla installation with the default language 'de-CH' or 'de-AT' or an another language which is not the most used language which is usually provided by extensions
• Manually add the <fallbackTag>de-DE</fallbackTag> to 'language/de-CH/langmetadata.xml' and 'administrator/language/de-CH/langmetadata.xml' in the <metadata> section use your language files descriptions and whatever fallback language you want to use the section looks like this afterwards:

<metadata>
                <name>German (Switzerland)</name>
		<nativeName>Deutsch (Schweiz)</nativeName>
		<tag>de-CH</tag>
		<fallbackTag>de-DE</fallbackTag>
		<rtl>0</rtl>
.
.
.

• Have an extension installed that does not provide your special language but that does provide the fallback language e.g. have de-CH or de-AT installed as default language and install a component which provides de-DE As extension you can use for example DPCalendar as it comes automatically shipped with the de-DE language.

Actual result BEFORE applying this Pull Request

Language falls back to English, best checked in the backend of the component

Expected result AFTER applying this Pull Request

Fallback language is used

Link to documentations

Please select:

• [ ] Documentation link for docs.joomla.org:

• [ ] No documentation changes for docs.joomla.org needed

• [ ] Pull Request link for manual.joomla.org:

• [ ] No documentation changes for manual.joomla.org needed

[MacJoom]

Sorry i wanted this PR for 4.3-dev - can anyone change this?

[coolcat-creations]

Means if a particullar string is not available in the main language it falls back to the fallback language? Or the whole file only?

[MacJoom]

Means if a particullar string is not available in the main language it falls back to the fallback language? Or the whole file only?

It tries to use an extensions language file as a possible fallback - if the extension provides a language file, it is loaded. this does not work on the string level

[brianteeman]

Would it not be better to enhance the current behaviour.

As it is today if the de-AT language file for the component does not exist then it falls back to en-GB

1. load de-AT
2. if not exist then load en-GB

I propose that instead the workflow is as follows

1. load de-AT
2. if not exist then load any language file beginning with de
3. if not exist then load en-GB

This way the "fallback" language does not have to be specified. It just follows a logical path

fr-CA --> fr-* --> en-GB

[MacJoom]

Would it not be better to enhance the current behaviour.

As it is today if the de-AT language file for the component does not exist then it falls back to en-GB

1. load de-AT
2. if not exist then load en-GB

I propose that instead the workflow is as follows

1. load de-AT
2. if not exist then load any language file beginning with de
3. if not exist then load en-GB

This way the "fallback" language does not have to be specified. It just follows a logical path

fr-CA --> fr-* --> en-GB

Thats a good point - we did indeed discuss an automatic solution, but i realized that this code was called very often and checking for an available language file may impact performance. On this level (not the string level) there is no fallback to en-GB (which wrongly would be en-EN) - Anyway i would like to change as little as possible - and there are not many countries needing this ;-)

[brianteeman]

Anyway i would like to change as little as possible - and there are not many countries needing this ;-)

exactly why I believe my proposal is the most efficient. (It's also the method used by other applications)

[laoneo]

Another question. When a de-DE file and de-CH file exists are always both of them loaded?

And please update the test instructions so other can test it as well. As extension you can use for example DPCalendar as it comes automatically shipped with the de-DE language.

[MacJoom]

Another question. When a de-DE file and de-CH file exists are always both of them loaded? Only the first available language is loaded, first language to test is the default language de-CH, then the fallback language. (See the rather ugly early return at line 717 - existing code)

[laoneo]

I have tested this item :white_check_mark: successfully on 64e89d9d808faee272d7e8fa424e10ae5a30d9e5

Tested it with DPCalendar as it comes with de-DE shipped. After installing the de-CH language I set the fallback language to de-DE in the metadata file of the site language and DPCalendar opened successfully in German. Before the patch it was English.

---

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/39152.}

[HLeithner]

I think it's the wrong direction. We should load a base language and override it with the localised Version.

[MacJoom]

I think it's the wrong direction. We should load a base language and override it with the localised Version.

You mean we should prepare (put in the array of files to be tried) automatically de-DE (derived from the first 2 characters in the language string) as well as the localised version (prior to the base version) without having the fallback tag? We could do that - it changes the behaviour of actual installations however

[laoneo]

I think it's the wrong direction. We should load a base language and override it with the localised Version.

Please add more information how you would do it. It is hard to understand what exactly you want to have differently. For the use case mentioned in this pr, this is a solid solution. Of course you can also do it like Brian mentioned but I like this approach more, so it is completely under control of the translation teams.

[ChrisHoefliger]

I have tested this item :white_check_mark: successfully on e9bd3a35186cd9fe1b96e2d513ad2afa72ee591b

I have tested this successfully on localhost with JCE both on site and in the backend.

---

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/39152.}

[laoneo]

RTC

---

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/39152.}

[HLeithner]

The thing is that normally you will install (we stay at the simple german or english version, based on my case) your local variant. This would be de-AT. the difference between de-AT and de-DE are only some marginal changes like ss and ß or Januar and Jänner. Now I have this shiny event component which has it's one translations for german and includes JANUARY as string which if we are optimistic is the only difference between de-DE and de-AT. So having all language string in de-AT a second time just for a single override is a type of overkill.

Other example why I think it's better to use the override approach is that your specific (or maybe own created) language file is out of date which means de-DE is already on 4.2.5 but de-AT is still on 4.1.5, this would mean for new strings it fallsback to english and not to the german german version.

Another part is this PR doesn't allow to fallback to de-DE from de-AT for joomla core.

Going the other way around could also be an option, the language pack can say which language tags it supports if no specific language is installed. de-DE would support:

• de-AT
• de-CH
• de-LI
• ...

if you look at english then you have about 10 or 20 country/region specific packages.

last but not least, why only one fallback language, in case of de-LI I may prefer de-AT if installed and if not use de-DE.

[MacJoom]

I think i should call the new tag <extensionFallbackTag>

[brianteeman]

@HLeithner that is exactly why I proposed the different approach

[laoneo]

Then @HLeithner and @brianteeman make a new pr so we can see the different approach. This one here fulfills the use case and is good for what it wants to improve.

[HLeithner]

Then @HLeithner and @brianteeman make a new pr so we can see the different approach. This one here fulfills the use case and is good for what it wants to improve.

that's your opinion, I don't think it's an improvement in the long run because so much more has to be fixed, just we have a pull request (thanks Martin) doesn't mean it have to be excepted (we talked about this on the maintainer sprint).

Merging this would prevent better/more complete solutions or at least creates more work on migration.

[MacJoom]

I don't think that this PR prevents better or more complete solutions. It is just a simple solution to a problem that special languages like de-CH have. One of the first things your hear in the swiss community is: Don't install de-CH as your favourite language because if you install extensions you will have them in english... (even if you have installed de-DE as an additional language and the extension provides de-DE)

[HLeithner]

de-AT has the same problem. As mentioned above, fall forward is better then fall back in this scenario. Load de-DE first and then if exists override with de-AT

[laoneo]

So having all language string in de-AT a second time just for a single override is a type of overkill.

But this is how Joomla works now and out of scope of this pr.

Other example why I think it's better to use the override approach is that your specific (or maybe own created) language file is out of date which means de-DE is already on 4.2.5 but de-AT is still on 4.1.5, this would mean for new strings it fallsback to english and not to the german german version. Another part is this PR doesn't allow to fallback to de-DE from de-AT for joomla core.

The problem is that a site doesn't have de-DE and de-AT installed at the same time, at least I see no reason to do so. So the different version of language pack issue doesn't count here and should be solved in a different pr when you want to fix this. This pr addresses a very common setup where people have only a variant like de-At installed but install an extension with only de-DE. All your other examples must be done in different pull requests and I doubt that this solution will block them in any way.

Going the other way around could also be an option, the language pack can say which language tags it supports if no specific language is installed.

Keep in mind an extension can ship it's language files in the installation bundle without any language manifest file. It is not an extra pack. Having this in the global language metadata definition, when a de-AT language also says, hey I'm de-CH as well, sounds for me like more trouble than it solves.

[HLeithner]

But this is how Joomla works now and out of scope of this pr.

it's simply the wrong approach from my point of view.

if you have a 10k lines files translation file for german and only 10 lines of austria dialect you think it's better to have 2x 10k lines. I don't think that this is right.

Also ignoring the core is not good, just saying out of scope makes no sense here, we have to check everything related.

[laoneo]

if you have a 10k lines files translation file for german and only 10 lines of austria dialect you think it's better to have 2x 10k lines. I don't think that this is right.

Again, this is not the use case this pr tries to solve. It would impact much more changes in the whole eco system that extension developers ship only overwrites for dialects. This would also require changes in translation systems like Transifex or crowdin and it is unlikely that this will happen in the closer future. So there is no reason to shut down this pr which solves a real world problem we are facing as extension developer and site admin.

I'm not against the approach from @brianteeman. But somebody needs to do that pr as alternative to this one. If this is not the case, then this here is enough and should be merged.

[brianteeman]

The main (only) difference with my approach is that the change is only made in this file and it just works.

This pr also only makes a change in this file BUT it requires all the language packs to be updated. If a language pack is not updated which is the main reason for this pr then the user will see the strings in english.

In summary. This pr does not solve the problem

[laoneo]

In the description is clearly mentioned that when this is merged, then the language packs are update afterwards.

[brianteeman]

Never mind, you clearly dont understand

[laoneo]

Perhaps you don't express your self clearly.

[brianteeman]

This pr also only makes a change in this file BUT it requires all the language packs to be updated. If a language pack is not updated which is the main reason for this pr then the user will see the strings in english.

[MacJoom]

I will do another PR without using the fallbackTag and loading the main language first