Initial support of region fully qualified Locale languages

[kevincianfarini]

This is a work in progress. I'm working through some issues on Android and JS. Right now, I'm striving to support fully qualified IETF locales -- language-COUNTRY-VARIANT.

I will post updates as I progress. Soon, I'll post what the build/moko directory looks like after these codegen changes.

Closes #329

[kevincianfarini]

Here's the structure of the generated folder so far. Some notes:

• Android requires that regional strings values folders be disambiguated with a r that precedes the Country code.
• JVM localizations require we use JVM Locale.toString format, eg. en_US rather than BCP en-US format.
• JS and iOS use BCP language format.

I've edited the tree for brevity. Let me know if you want the raw output of tree.

sample/mpp-library/build/generated/moko/
├── androidMain
│   ├── res
│   │   ├── values
│   │   │   ├── multiplatform_plurals.xml
│   │   │   └── multiplatform_strings.xml
│   │   ├── values-en-rGB
│   │   │   └── multiplatform_strings.xml
│   │   ├── values-en-rUS
│   │   │   └── multiplatform_strings.xml
│   │   ├── values-es
│   │   │   ├── multiplatform_plurals.xml
│   │   │   └── multiplatform_strings.xml
│   │   ├── values-night
│   │   │   └── colors.xml
│   │   └── values-ru
│   │       ├── multiplatform_plurals.xml
│   │       └── multiplatform_strings.xml
├── iosSimulatorArm64Main
│   ├── res
│   │   ├── Base.lproj
│   │   │   ├── Localizable.strings
│   │   │   └── Localizable.stringsdict
│   │   ├── en-GB.lproj
│   │   │   └── Localizable.strings
│   │   ├── en-US.lproj
│   │   │   └── Localizable.strings
│   │   ├── es.lproj
│   │   │   ├── Localizable.strings
│   │   │   └── Localizable.stringsdict
│   │   ├── ru.lproj
│   │   │   ├── Localizable.strings
│   │   │   └── Localizable.stringsdict
├── jsMain
│   ├── comicerockdevlibrary
│   │   └── res
│   │       └── localization
│   │           ├── comicerockdevlibrary_pluralsJson.json
│   │           ├── comicerockdevlibrary_pluralsJson_es.json
│   │           ├── comicerockdevlibrary_pluralsJson_ru.json
│   │           ├── comicerockdevlibrary_stringsJson.json
│   │           ├── comicerockdevlibrary_stringsJson_en-GB.json
│   │           ├── comicerockdevlibrary_stringsJson_en-US.json
│   │           ├── comicerockdevlibrary_stringsJson_es.json
│   │           ├── comicerockdevlibrary_stringsJson_ru.json
│   │           ├── comicerockdevlibrarynested_stringsJson.json
│   │           └── comicerockdevlibrarynested_stringsJson_ru.json
├── jvmMain
│   ├── assets
│   ├── comicerockdevlibrary
│   │   └── res
│   │       └── localization
│   │           ├── comicerockdevlibrary_mokoBundle.properties
│   │           ├── comicerockdevlibrary_mokoBundle_en_GB.properties
│   │           ├── comicerockdevlibrary_mokoBundle_en_US.properties
│   │           ├── comicerockdevlibrary_mokoBundle_es.properties
│   │           ├── comicerockdevlibrary_mokoBundle_ru.properties
│   │           ├── comicerockdevlibrary_mokoPluralsBundle.properties
│   │           ├── comicerockdevlibrary_mokoPluralsBundle_es.properties
│   │           └── comicerockdevlibrary_mokoPluralsBundle_ru.properties
├── macosArm64Main
│   ├── assets
│   ├── res
│   │   ├── Base.lproj
│   │   │   ├── Localizable.strings
│   │   │   └── Localizable.stringsdict
│   │   ├── en-GB.lproj
│   │   │   └── Localizable.strings
│   │   ├── en-US.lproj
│   │   │   └── Localizable.strings
│   │   ├── es.lproj
│   │   │   ├── Localizable.strings
│   │   │   └── Localizable.stringsdict
│   │   ├── ru.lproj
│   │   │   ├── Localizable.strings
│   │   │   └── Localizable.stringsdict

[kevincianfarini]

Implementation notes: Right now, if my default language is en for MR/Base, and I specify a folder of MR/en-rUS, on Android this setup won't fall back to Base if my language is set to simply en. It finds a partial match of en in en-US and goes with it.

This causes some failing tests on the android build in StringResourceTests.kt. I'm hoping for some guidance here on what you think the best outcome might be.

[kevincianfarini]

Thanks for the initial review @Alex009. Everything is resolved aside from the failing tests in androidMain. I'm dedicating today to this issue. 🚀

[kevincianfarini]

@Alex009 The most recent commit should have resolved detekt issues. Mind running CI again?

[kevincianfarini]

@Alex009 Ping! Is there anything else I should to before we run CI again?

[Alex009]

@kevincianfarini sorry, i miss message. i run CI now

[kevincianfarini]

One more CI run please 😅

[kevincianfarini]

Alright....one last time. Resolved JVM file name conflict.

[kevincianfarini]

Seems like the most recent CI failure was a flake?

 > Could not determine the dependencies of null.
See https://docs.gradle.org/7.4.2/userguide/command_line_interface.html#sec:command_line_warnings
   > Could not resolve all task dependencies for configuration ':classpath'.
      > Could not resolve com.android.tools.build:gradle:7.2.0.
        Required by:
            project : > project :resources-build-logic
         > Could not resolve com.android.tools.build:gradle:7.2.0.
            > Could not get resource 'https://repo.maven.apache.org/maven2/com/android/tools/build/gradle/7.2.0/gradle-7.2.0.pom'.
               > Could not GET 'https://repo.maven.apache.org/maven2/com/android/tools/build/gradle/7.2.0/gradle-7.2.0.pom'.
                  > repo.maven.apache.org
         > Could not resolve com.android.tools.build:gradle:7.2.0.
            > Could not get resource 'https://dl.google.com/dl/android/maven2/com/android/tools/build/gradle/7.2.0/gradle-7.2.0.pom'.
               > Could not GET 'https://dl.google.com/dl/android/maven2/com/android/tools/build/gradle/7.2.0/gradle-7.2.0.pom'.
                  > dl.google.com

[Reedyuk]

@Alex009 - any chance you could push this through? i also would like to use this feature as its a red flag for me going forward.

[Alex009]

@Reedyuk i think we release this changes in this month.

@kevincianfarini now CI failed because of android lint:

/Users/runner/work/moko-resources/moko-resources/sample/mpp-conditional/build/generated/moko/androidMain/res/values/multiplatform_strings.xml:3: Error: "common_name" is not translated in "en" (English) [MissingTranslation]
 <string name="common_name">Test Project</string>
         ~~~~~~~~~~~~~~~~~~
The full lint text report is located at:
  /Users/runner/work/moko-resources/moko-resources/sample/mpp-conditional/build/intermediates/lint_intermediate_text_report/debug/lint-results-debug.txt

[kevincianfarini]

@Alex009 @Reedyuk I hope to get to this today. Currently working on a production issue at work.

[Reedyuk]

@Alex009 @Reedyuk I hope to get to this today. Currently working on a production issue at work.

Thanks Kevin, i didn't realise Octopusenergy was using KMM, nice job

[kevincianfarini]

@Alex009 I'm unable to reproduce this lint failure locally. Can you download the lint report and paste the relevant bits here?

[kevincianfarini]

Related to this unit of work, but likely outside of the scope of this PR. I think there's a bug with asset resolution in JS and possibly darwin. The following test.

    @Test fun stringFallsBackToLanguageUnspecifiedDialect() = runTest {
        StringDesc.localeType = StringDesc.LocaleType.Custom("es-US")
        val rawString: String = MR.strings.test_simple.desc().getString()
        assertEquals(
            expected = "prueba",
            actual = rawString
        )
    }

fails to resolve to the es language on JS and iOS. I've added this test to the master branch which still fails.

The MDN docs for Navigator.language specifies that it can be a full BCP 47 language tag, eg. en-US.

https://developer.mozilla.org/en-US/docs/Web/API/Navigator/language

I can create a follow up issue if you'd like!

[kevincianfarini]

@Alex009 I'm unable to reproduce this lint failure locally. Can you download the lint report and paste the relevant bits here?

Ping!

[kevincianfarini]

@Alex009 Following up. I am still unable to reproduce this lint error. Can you paste the full lint logs?

[kevincianfarini]

@Alex009 Following up! I'm still unable to reproduce the lint error locally. Can you please copy/paste the CI error logs so I can resolve?

[Alex009]

@kevincianfarini sorry for late reaction :( there all that i have - logs_604.zip

[kevincianfarini]

@Alex009 No worries! Thanks for grabbing those for me.

[kevincianfarini]

@Alex009 I need the following file, which should have 12 lint errors, and 14 warning.

file:///Users/runner/work/moko-resources/moko-resources/sample/mpp-conditional/build/reports/lint-results-debug.html

The above linked logs don't don't have all of the info I need.

2022-08-11T01:34:16.9948750Z Lint found 12 errors, 14 warnings. First failure:
2022-08-11T01:34:16.9948930Z 
2022-08-11T01:34:16.9949540Z /Users/runner/work/moko-resources/moko-resources/sample/mpp-conditional/build/generated/moko/androidMain/res/values/multiplatform_strings.xml:3: Error: "common_name" is not translated in "en" (English) [MissingTranslation]
2022-08-11T01:34:16.9950160Z  <string name="common_name">Test Project</string>
2022-08-11T01:34:16.9950410Z          ~~~~~~~~~~~~~~~~~~
2022-08-11T01:34:17.0051220Z 
2022-08-11T01:34:17.0152280Z The full lint text report is located at:
2022-08-11T01:34:17.0253930Z   /Users/runner/work/moko-resources/moko-resources/sample/mpp-conditional/build/intermediates/lint_intermediate_text_report/debug/lint-results-debug.txt

It only includes the first failure. Sorry this is so tedious for you!

[kevincianfarini]

@Alex009 Also, if you wouldn't mind trying to get this lint task to fail locally for you that would be great. This is the offending one on CI, but it builds successfully on my machine.

./gradlew :sample:mpp-conditional:lintDebug

[kevincianfarini]

We could also just baseline these lint errors if you're open to that.

[filippodelfra]

Hi guys, is there any chance to get this pr merged? Thanks

[kevincianfarini]

I've had other priorities pop up and won't be able to allocate time to this. I would be happy for @Alex009 to take it, resolve the lint issue, and merge.

[ScottPierce]

Getting this in soon would be great!

[github-advanced-security[bot]]

You have successfully added a new SonarCloud configuration ``. As part of the setup process, we have scanned this repository and found no existing alerts. In the future, you will see all code scanning alerts on the repository Security tab.