Message context for Yii::t()

[Deele]

I feel that Yii::t() lacks ability to provide context of textual value and when translations start to count in couple hundreds, it starts to be quite hard to understand what was meant with translation.

If we look at popular translation systems like "Crowdin" (example), they allow to provide message context to support translators.

Currently, only way to provide some helpful information using Yii::t() is to write something in "key" area, that in turn defies purpose of "source language", for example, if you write a message "Propose (button in issue tasklist table header)" until I change that translation to "Propose", that message still be visible.

I am guessing, that there is performance issue with long keys, and length limitations does not allow long context descriptions.

For backwards compatability, I would like to suggest 5th optional attribute to Yii::t($category, $message, $params = [], $language = null, $context = null) (but this is not very convenient) that would allow to provide some sort of context description.

Maybe, we could convert Yii::t() to allow passing in first attribute array of attributes, like Yii::t(['category' => 'message.category', 'message' => 'message text', 'params' => [], 'language' => null, 'context' => null]) or even allow shorthand for category and message with numeric keys Yii::t(['message.category', 'message text', 'params' => [], 'language' => null, 'context' => null]).

With \yii\i18n\DbMessageSource, they would be stored in separate table source_message__context, where we would store source message ID and context description with 255 characters long string.

With \yii\i18n\PhpMessageSource, they would be stored in separate file, right next to current message files, in {category}_context.php format, where we would have associative array with source message category as a key, and array with context descriptions, as a value.

I have no experience with \yii\i18n\GettextMessageSource, but I see in their documentation, that they already have an option to provide context descriptions. But I would like to see some suggestions for this solution.

This feature would allow to display all of those context description together with message in interface, to help translator to translate messages and understand context of it.

[samdark]

Getext context is what we know as a category, not an example of usage: https://www.gnu.org/software/gettext/manual/html_node/Contexts.html

[klimov-paul]

I am against such feature. Placing some descriptive information around any call of Yii::t() sounds like overkill to me.

Besides how we can control same pair of 'category' and 'message' will have same 'context' at all its invocations? For example:

Yii::t('app', 'Some message', [], 'en', 'Some context');
// ...
Yii::t('app', 'Some message', [], 'en', 'Another context!!!');

Each translation message has unique key, which is a combination of category and message. If you wish to add some description to them you can use this pair and your own custom wrapper.

[samdark]

Agree.

[Deele]

@klimov-paul I don't see arguments about such approach being overkill. I already explained how to control/store them. Each message can have multiple contexts as in your example. Nothing changes regarding identifiers.

[samdark]

@Deele

1. Not all formats we support allowing for such feature. GetText does not.
2. There's category to differentiate contexts.
3. You have to preview your translation somehow anyway. It's not possible to translate 100% perfectly w/o checking actual UI usage because of many factors: context which could not be effectively described in text, UI specifics etc.

[Renkas]

1. GetText supports comments: https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html
2. it's not very descriptive. A freetext comment can be usefull sometimes.
3. Of course - but translators don't have that possibility usually (at the time of translation).

[samdark]

OK. If comments are supported in GetText we may introduce it.

[samdark]

Still, I don't think these descriptions are good to be placed right in the code.

[klimov-paul]

I don't see arguments about such approach being overkill.

For me internal program code should not contain such information since it does not influence the program logic. You propose to make source code, which will be executed on each(!) program run much heaiver to support translation functionality, which is performed relatively rare.

Yii::t() function serves the functioality purpose, while yii message command is a service. I do not like the idea to mix up functionality with the service.

Each message can have multiple contexts as in your example

And how it should look like in the end? How it should be stored? What it should mean for the interpreter who will work with translations?

One more thing. What you are planning to do in case your interface for the translations management should also provide i18n feature? This will mean that your 'context' should be translated as well. Otherwise it make not much sense then a plain message ID.

[klimov-paul]

GetText supports comments:

GetText instroduce sperated file per each language. So you mean 'context' may vary per each language in your case? As far I can see 'context' is something universal through all language translations. Saving context inside GetText will just cause a lot of data duplication.

[Renkas]

Don't confuse comments with context/category.

[SilverFire]

With \yii\i18n\PhpMessageSource, they would be stored in separate file, right next to current message files, in {category}_context.php format, where we would have associative array with source message category as a key, and array with context descriptions, as a value.

I read this and can't find points why the aforementioned is worth than creating a separate dictionary for a specific context. I see the following cons:

• New entity
• More code to support
• More complexity to Yii::t API (passing 4-th argument after two optional)
• More complexity to messages resolving
• Need to re-write yii message, as @klimov-paul said in his last message

Therefore I'm standing against this feature request.

[Renkas]

xgettext utility works like so:

Comments (starting with ///) placed directly before strings thus marked are made available as hints to translators by helper programs. https://en.wikipedia.org/wiki/Gettext#Programming

I see the problem having comment in a method call. Couldn't the message parser support some kind of comments markup?

/// Some hint for translators here
echo Yii::t('app', 'Awesome string');

And for inline usage maybe the parser could be smart enough to recognise comment that is inserted immediately after method call?

echo '<b>' . Yii::t('app', 'Awesome string')/* Some hint for translators here */ . '</b>';

Of course the parser should then be able to work when there are two method calls and hints on same line (example for test case ;) ):

echo '<b>' . Yii::t('app', 'Awesome string')/* Some hint for translators here */ . '</b> - <b>' . Yii::t('app', 'Another awesome string')/* Hint for the other string */ . '</b>';

[Renkas]

@SilverFire so you dislike the idea because it means someone needs to re-write yii message? Or why did you down vote without any comments? All other points from your earlier post has been addressed in this solution.

Anyway I don't really like the opening and closing of this thread in a short amount of time without waiting for answers to open questions. And down voting from Yii core developer without giving a reason after all the issues have been addressed is a very odd move.

[rob006]

WordPress uses such approach. https://developer.wordpress.org/themes/functionality/internationalization/#descriptions https://develop.svn.wordpress.org/trunk/tools/i18n/extract.php

[SilverFire]

Or why did you down vote without any comments

I was disturbed while writing a comment. Then found, that I've confused translators comments and context. :unamused: @Renkas sorry

[klimov-paul]

Anyway I don't really like the opening and closing of this thread in a short amount of time without waiting for answers to open questions. And down voting from Yii core developer without giving a reason after all the issues have been addressed is a very odd move.

We have provided all necessary explanations. We consider this feature is not worth the overcomlication of the code (both yii2 core internal and possible actual project one). We can not accept such changes into the main codebase in near future.

However, you can implement this feature as a separated extension outside the yiisoft scope and use it as you pleased.

[Renkas]

@klimov-paul I don't think you read the last posts here ... There's exactly 0 changes to Yii core code. Only thing that needs changing is message extraction and some documentation

[klimov-paul]

So you consider yii\console\controllers\MessageController is not a part of Yii2???

[Renkas]

I haven't said that. If it would not be a part of Yii then there would be no point for this thread would it?

But it's not part of core code as you said - core code would run perfectly fine without it. It's an utility - and quite useful one (at least for me). Support for message hints (should be the proper name for this feature I think?) is currently missing from it. And I don't really see why it could not be a part of the default functionality.

Right now it seems you just try to defend the decision on why the issue is closed and are not even thinking about the new proposal. That's not a very productive way of improving anything (which is what software development is all about I think?).

Anyway I'm not going to debate you on this anymore. If Yii team has made up it's mind then so be it. If I really need it I'll implement it myself - just seems like a thing that would probably be useful for lot's of developers.

[kidol]

@Renkas Can't you use actual message as descriptive context? What I mean is, set sourceLanguage to some dummy value, then run message command and translate to language.

'Welcome {user} # some context' => "Welcome {user}"`

[Renkas]

Of course this is a possible hack and as far as I have seen from forum threads etc some people use it. But it's still kind of a hack.

My projects are all using proper base language as strings as I'm not a fan of this dummy language solution. But that's just my preference ...

[samdark]

I'd like to discuss it a bit more after some thinking. Adding more functionality to message parser isn't a big problem. The problem is not to complicate default usage which doesn't have this context.

Using comments for the purpose looks a bit like hacking to me and, as @klimov-paul said, it adds load to APC/OpCache requiring more memory to store these.

@Deele, @Renkas do you want these comments to appear strictly in the code or you just want them to be there to be filled later?

[Renkas]

Me personally would like the possibility to write some hints/explanations to the translators when needed. In my use case I use parser to generate gettext files that are then synced to external translation service and then synced back to my application (overwriting the parsed .po files).

So what I want is a mechanism for the message parser to get these message hints from my application source code and save it to the generated .po file - which will be synced to the external service and the hints will be available there for translators.

• These hints are in itself an exception - would only use when really needed - not everywhere.
• One problem raised before was if the same message has multiple different hints/comments in different places in code.
  • My solution would be to append all of them (dividing them clearly of course).
  • When same string used in multiple places has hint defined only in one place then only that would be used - and there would be no problem
• Load on APC/OpCache part seems not to be a real problem. In PHP source you have way more comments anyway. If you mean caching .po files then yes there would be some extra lines - but I think that's quite marginal.

Of course there are other message sources to think about. In PHP message source I think they could be used as PHP comments as well? And in DB message source maybe add a extra column?

[Deele]

The message and category are key fields with limited length and other limitations. I don't want to disturb this behavior, that is why putting more information in category or the message does not really work. I want separate field, that is not keyed and indexed and could be provided within code.

I agree to @klimov-paul that there is a problem of overhead because of descriptions that are parsed in every code execution. That could be handled with use of comments as @Renkas proposed, but that could become ugly for multiple inline Yii::t() calls and even more ugly for parser to recognize.

If it is not possible to provide context description within code itself, then this feature should be handled separately altogether without changes to Yii::t() with separate table/model/solution (using message/category as the key) and this specific discussion is over.

Another idea is to gather and make available other kind of message context information - where exactly in code that specific Yii::t() is called, so that in interface, it would look like list of file names, class names, method names, line numbers and other information (to be able to create a URL to create clickable reference to repository, if application has one), that could be used by technically advanced translators, to use source code as a guide. This is not really user friendly approach, but could be much easier to implement and would not require any changes to Yii::t() syntax.

[samdark]

I'm against expanding method signature but comments approach looks at least acceptable:

1. It won't affect code execution much if not used everywhere extensively (number of comments affects how much is saved into memory from APC/OpCache).
2. It will be backwards compatible.
3. It would require to change only messages table and the message console command.

Sounds like a plan to me.

@Renkas, @Deele, is that what you want?

[samdark]

@klimov-paul, @SilverFire the approach above seems to not complicate framework much except console command message parser.

[klimov-paul]

The main problem here is a extracted context should be saved somewhere. Each particular message storage should provide a way to store it.

For the GetText its internal comments feature is proposed to be used. For the PHP and DB it should be defined.

As we have already found out, each message may have several contexts. Thus context should be stored appropriately. For example: for DB there should be an extra table, which will reference as 'many-to-one' to the message table.

[Renkas]

@samdark yes that would cover my needs

[kidol]

How will it help a human translator if a unique message has several (different) comments attached to it? I don't get it.