feat: New command `lang:sync`

[neznaika0]

Description A simple command to copy existing translation files to a new locale. From the file Language/en/Example.php:

<?php

return [
    'title' => 'Default title',
    'status' => [
        'error'    => 'Error!',
        'done'     => 'Done!',
        'critical' => 'Critical!',
    ],
    'nullableKey' => null,
    'numericKey' => 100500,
    'more' => [
        'nested' => [
            'key' => 'More nested key...',
        ],
    ],
];

After the command php spark lang:sync --locale en --target ru we get Language/ru/Example.php .

<?php

return array (
  'more' => 
  array (
    'nested' => 
    array (
      'key' => 'Example.more.nested.key',
    ),
  ),
  'nullableKey' => 'Example.nullableKey',
  'numericKey' => 'Example.numericKey',
  'status' => 
  array (
    'error' => 'Example.status.error',
    'done' => 'Example.status.done',
    'critical' => 'Example.status.critical',
  ),
  'title' => 'Example.title',
);

applied cs-fix:

<?php

return [
    'more' => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
    'nullableKey' => 'Example.nullableKey',
    'numericKey'  => 'Example.numericKey',
    'status'      => [
        'error'    => 'Example.status.error',
        'done'     => 'Example.status.done',
        'critical' => 'Example.status.critical',
    ],
    'title' => 'Example.title',
];

And so it is with all other translations

=====

When there are translations in the target folder Language/ru, their keys are saved only if they exist in the Language/en folder:

Language/ru/Example.php:

<?php

return [
    // Deleted keys
    // 'more' => [
    //     'nested' => [
    //         'key' => 'Example.more.nested.key',
    //     ],
    // ],
    'nullableKey' => null,
    'numericKey'  => 100000,
    'status'      => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'title' => 'ru.title',
    'mistakeKey' => 'ru.mistake_key',
];

Language/ru/Example.php after sync:

<?php

return [
    'more' => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
    'nullableKey' => null,
    'numericKey'  => 100000,
    'status'      => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'title' => 'ru.title',
];

Checklist:

• [x] Securely signed commits
• [x] Component(s) with PHPDoc blocks, only if necessary or adds value
• [ ] Unit testing, with >80% coverage
• [x] User guide updated
• [x] Conforms to style guide

[neznaika0]

There are some questions:

1. Does anyone else need it? :smile: I have indicated the reason for using it as a more precise translation control.

2. I need help with the documentation. I don't know English

3. Formatting new arrays requires the use of php-cs-fixer. This was discussed in another PR https://github.com/codeigniter4/CodeIgniter4/pull/7896#issuecomment-1712702238 . How can we take this out for general use?

4. phpstan. I don't quite understand what he wants from me. I have specified a common type for the values

[neznaika0]

Userguide:

Changelog:

[kenjis]

I don't get the description "When there are translations in the target folder Language/ru, their keys are saved only if they exist in the Language/en folder:". Is that correct?

If app/Language/en/Example.php is:

return [
    // Deleted keys
    // 'more' => [
    //     'nested' => [
    //         'key' => 'Example.more.nested.key',
    //     ],
    // ],
    'nullableKey' => null,
    'numericKey'  => 100000,
    'status'      => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'title' => 'ru.title',
    'mistakeKey' => 'ru.mistake_key',
];

Then, app/Language/ru/Example.php will be:

return [
    'mistakeKey'  => 'Example.mistakeKey',
    'nullableKey' => 'Example.nullableKey',
    'numericKey'  => 'Example.numericKey',
    'status'      => [
        'error'    => 'Example.status.error',
        'done'     => 'Example.status.done',
        'critical' => 'Example.status.critical',
    ],
    'title' => 'Example.title',
];

That is, deleted keys are gone.

[kenjis]

If app/Language/en/Example.php is:

return [
    'title'  => 'Default title',
    'status' => [
        'error'    => 'Error!',
        'done'     => 'Done!',
        'critical' => 'Critical!',
    ],
    'nullableKey' => null,
    'numericKey'  => 100500,
    'more'        => [
        'nested' => [
            'key' => 'More nested key...',
        ],
    ],
];

Then, app/Language/ru/Example.php will be:

return [
    'more' => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
    'nullableKey' => 'Example.nullableKey',
    'numericKey'  => 'Example.numericKey',
    'status'      => [
        'error'    => 'Example.status.error',
        'done'     => 'Example.status.done',
        'critical' => 'Example.status.critical',
    ],
    'title' => 'Example.title',
];

That is, the key order changed. This makes comparison of the two files, original and translated, difficult.

[kenjis]

This command would be useful. However, I would like the order of the keys to match the original.

[neznaika0]

I don't get the description "When there are translations in the target folder Language/ru, their keys are saved only if they exist in the Language/en folder:". Is that correct?

If app/Language/en/Example.php is: ...

No. Look at the order of the files. We from EN combine RU. There are translations in RU ru.title, so the keys are saved and not overwritten Example.title. Old keys remain.

That is, the key order changed. This makes comparison of the two files, original and translated, difficult.

Yes. The order has changed. I'm sorting a new array. I think it can be fixed. But the keys will be chaotic when combined.

[kenjis]

This PR branch is too old. Can you rebase?

[kenjis]

No. Look at the order of the files. We from EN combine RU. There are translations in RU ru.title, so the keys are saved and not overwritten Example.title. Old keys remain.

Okay, I got what you say.

app/Language/en/Example.php:

return [
    'title'  => 'Default title',
    'status' => [
        'error'    => 'Error!',
        'done'     => 'Done!',
        'critical' => 'Critical!',
    ],
    'nullableKey' => null,
    'numericKey'  => 100500,
    'more'        => [
        'nested' => [
            'key' => 'More nested key...',
        ],
    ],
];

app/Language/ru/Example.php:

<?php

return [
    // Deleted keys
    // 'more' => [
    //     'nested' => [
    //         'key' => 'Example.more.nested.key',
    //     ],
    // ],
    'nullableKey' => null,
    'numericKey'  => 100000,
    'status'      => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'title' => 'ru.title',
    'mistakeKey' => 'ru.mistake_key',
];

$ php spark lang:sync --locale en --target ru
$ composer cs-fix

app/Language/ru/Example.php:

return [
    'more' => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
    'nullableKey' => null,
    'numericKey'  => 100000,
    'status'      => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'title' => 'ru.title',
];

[kenjis]

Yes. The order has changed. I'm sorting a new array. I think it can be fixed. But the keys will be chaotic when combined.

I think the key order should not be changed.

Can't you rewrite with recursive calls? This is PoC sample code by ChatGPT:

<?php

function syncTranslationsRecursive($original, $translations) {
    // Initialize an array to store the results
    $syncedTranslations = [];

    // Loop through the original message array
    foreach ($original as $key => $message) {
        if (is_array($message)) {
            // If the message is an array, recursively synchronize it
            $syncedTranslations[$key] = syncTranslationsRecursive(
                $message,
                array_key_exists($key, $translations) ? $translations[$key] : []
            );
        } else {
            // If the message is not an array, check if the translations array contains the same key
            if (array_key_exists($key, $translations)) {
                $syncedTranslations[$key] = $translations[$key];
            } else {
                // If the key doesn't exist in the translations array, set an empty string
                $syncedTranslations[$key] = '';
            }
        }
    }

    return $syncedTranslations;
}

// Original message array
$original = [
    'title' => 'Default title',
    'status' => [
        'error'    => 'Error!',
        'done'     => 'Done!',
        'critical' => 'Critical!',
    ],
    'more' => [
        'nested' => [
            'key' => 'More nested key...',
        ],
    ],
];

// Translated message array (partially translated)
$translations = [
    'title' => 'Título predeterminado',
    'status' => [
        'error' => '¡Error!',
        'done'  => '¡Hecho!',
    ],
];

// Synchronize the translations array with the original message array
$syncedTranslations = syncTranslationsRecursive($original, $translations);

// Print the results
print_r($syncedTranslations);

[neznaika0]

What is the disadvantage of the current array merge?

https://github.com/neznaika0/codeigniter-dev/blob/19f9440bd9deb5131079ff427781f6f14de99584/system/Helpers/Array/ArrayHelper.php#L327-L341

I can just remove the sorting and everything looks like you wanted. https://github.com/neznaika0/codeigniter-dev/blob/19f9440bd9deb5131079ff427781f6f14de99584/system/Commands/Translation/LocalizationSync.php#L136-L137

But, if the target file has keys, then they are added to the end.

[kenjis]

I would like the key order is never changed in any case. If so, I don't care about the implementation.

[neznaika0]

Results based on the data above:

// translation RU is empty
return [
    'title'  => 'Example.title',
    'status' => [
        'error'    => 'Example.status.error',
        'done'     => 'Example.status.done',
        'critical' => 'Example.status.critical',
    ],
    'nullableKey' => 'Example.nullableKey',
    'numericKey'  => 'Example.numericKey',
    'more'        => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
];

// translation RU exist
return [
    'nullableKey' => null,
    'numericKey'  => 100000,
    'status'      => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'title' => 'ru.title',
    'more'  => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
];

[kenjis]

When translation RU exist, the key order will be different from EN. In any case, the key order should be the same in all lang files.

[neznaika0]

// Before: app/Language/en/Example.php
return [
    'title'  => 'Default title',
    'status' => [
        'error'    => 'Error!',
        'done'     => 'Done!',
        'critical' => 'Critical!',
    ],
    'nullableKey' => null,
    'array'       => [],
    'numericKey'  => 100500,
    'more'        => [
        'nested' => [
            'key' => 'More nested key...',
        ],
    ],
];

// Before: app/Language/ru/Example.php
return [
    // Deleted keys
    // 'more' => [
    //     'nested' => [
    //         'key' => 'Example.more.nested.key',
    //     ],
    // ],
    'nullableKey' => null,
    'status'      => [
        'critical' => 'ru.status.critical',
        'done'     => 'ru.status.done',
        'error'    => 'ru.status.error',
    ],
    'mistakeKey' => 'ru.mistake_key',
    'numericKey'  => 100000,
    'title' => 'ru.title',
];

// After: app/Language/ru/Example.php
// 1. Right sort as in EN
// 2. Delete 'mistakeKey' (in EN language not exist)
// 3. Untranslated keys have a placeholder ''Example.more.nested.key'
// 4. Previously translated keys are saved
return [
    'title'  => 'ru.title',
    'status' => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'nullableKey' => null,
    'array'       => [
    ],
    'numericKey' => 100000,
    'more'       => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
];

The array intersection search function ArrayHelper::intersectKeyRecursive is no longer needed. Combining arrays should now initially not have extra keys (current mistakeKey) from the target array.

[datamweb]

I prefer the following result. To make it easier for the developer to translate the file.

// After: app/Language/ru/Example.php
return [
    'title'  => '(To be translated) Default title',
    'status' => [
        'error'    => '(To be translated) Error!',
        'done'     => '(To be translated) Done!',
        'critical' => '(To be translated) Critical!',
    ],
    'nullableKey' => null,
    'array'       => [
    ],
    'numericKey' => 100000,
    'more'       => [
        'nested' => [
            'key' => '(To be translated) More nested key...',
        ],
    ],
];

[neznaika0]

@datamweb I think it's wrong. If the current locale is RU and the translations are the same (EN), it is unclear on the page whether the translation was made or not. Because the fallback translation is EN.

Additional text interferes with the preview of the page (too long for some places).

Placeholders clearly show untranslated lines.

It is possible to make it configurable --replace-old

[michalsn]

Okay, what is the status of this feature?

It seems like the order of the keys is updated.

Personally, I don't think we need the (To be translated) part as differences in languages are usually quite obvious.

Are there any remaining issues?

[neznaika0]

I'll update the PR in a week. From the corrections: Checking the value is a string. In general, we can abandon PR, since I have a separate repository for lang:find. Synchronization can be moved there.

[michalsn]

In general, we can abandon PR, since I have a separate repository for lang:find. Synchronization can be moved there.

So you would rather add this function to the existing lang:find command? Do I understand this correctly?

[neznaika0]

If this command is not needed in CI4, I can add it to the optional package. The command is separate from lang:find, but in the same group Translation

[neznaika0]

Ready.

[neznaika0]

Ready.

• updated igroring files - сhecking extension is enough
• added EXIT_ and process() returns an int
• added new test for checking language folders
• ErrorException is added because the bool for mkdir() is unreliable

[neznaika0]

Thanks to everyone who helps.

[samsonasik]

Thank you @neznaika0