Updating comment/note in source template file does not correctly update comment/note in translations

[henry-torproject]

Describe the issue

Seen when using Fluent, but I imagine it would effect other formats that use a template.

Whenever the source file changes the comment/note on one of its strings, the translation files do not pick up on this change, and Weblate shows the "string description" as a concatenation of the old and new strings.

I already tried

• [X] I've read and searched the documentation.
• [X] I've searched for similar filed issues in this repository.

Steps to reproduce the behavior

Tested with my local weblate instance, but I've seen this for the Tor Project before:

1. Have a Fluent component with a template file "en.ftl" containing a Fluent string with a comment "Original comment.", and translate it to "de.ftl".
2. Change the comment in the locally stored file "en.ftl" to be "New comment.".
3. Force weblate to pick up the new changes.

Weblate will show the en string with the string description: "New comment.". But the de string will have the string description: "New comment. Original Comment.". Moreover, the "de.ftl" file will only contain the old comment: "Original Comment.".

NOTE: The concatenation comes from this line: https://github.com/WeblateOrg/weblate/blob/a808cf013882ae41987921cf93301c0055e3cdcc/weblate/formats/ttkit.py#L106-L110 and seems to be a result of the translated file not picking up on the comment change.

Expected behavior

If the notes on a template unit changes, then all translation units with the same original note should update their note to match the new value. I.e. the new comment should entirely replace the old comment on Weblate, and in the translated files.

Screenshots

No response

Exception traceback

No response

How do you run Weblate?

weblate.org service

Weblate versions

No response

Weblate deploy checks

No response

Additional context

No response

[nijel]

Are actually comments in the translated file desired? Keeping them in sync is an additional effort which would have to be performed on each template file update.

[henry-torproject]

Are actually comments in the translated file desired? Keeping them in sync is an additional effort which would have to be performed on each template file update.

@nijel I think for tor project's use case, no comments in the translated files would function ok, and I'm not sure we ever really read the comments in the translated files anyway.

But I'm not sure for other projects, since as far as I can tell this bug would also effect other template-based formats besides Fluent.

[nijel]

I'm not 100% sure how all the formats work, but there is definitely no synchronization in place.

From what I know:

• Some formats discard the comments while adding strings to the translation. This applies to all LISA based formats which store content in XML tree and not in the translate-toolkit Unit (Android, TS2 or XLIFF).
• There are formats which do not support comments at all (plain JSON) or in translate-toolkit (YAML).
• Others which store info on the translate-toolkit unit will probably preserve comments (Properties, Fluent, some JSON variants as WebExtension).

Possible solutions:

• Remove comments completely when adding new strings to translated files. This avoids synchronization issues, but will look weird in some formats (eg. WebExtension JSON).
• Sync comments while translating. This would help in situations where both source and comment change.
• Implement add-on to sync comments on template file update. This is a similar approach to https://docs.weblate.org/en/latest/admin/addons.html#update-po-files-to-match-pot-msgmerge.

[github-actions[bot]]

This issue has been put aside. It is currently unclear if it will ever be implemented as it seems to cover too narrow of a use case or doesn't seem to fit into Weblate.

Please try to clarify the use case or consider proposing something more generic to make it useful to more users.

[gzm0]

We are seeing the same issue with Java property files. A sequence that is highly problematic for us is as follows:

tl;dr comment gets put to target file but never updated afterwards

1. New key gets added (by devs) in base file including a comment describing the key. For example:

   # user alert when frobber happened. 
   the_alert=Frobber just occurred!
   

2. Weblate loads it (uses the comment as "source string description", awesome!), translators translate it.

3. Weblate commits the translations including the comment. For example:

   # user alert when frobber happened. 
   the_alert=Frobber ist passiert!
   

   It is worth pointing out that neither the devs nor the translators need these comments (but do not mind them either).

4. The devs update the translation and comment (in the base file):

   -# user alert when frobber happened. 
   -the_alert=Frobber just occurred!
   +# user alert when flibber happened. 
   +the_alert=Flibber just occurred!
   

5. Weblate loads it and flags translations as needs-review (awesome!).

6. Translators see the concatenated source string description. For example:

   user alert when flibber happened. user alert when frobber happened.

   That's a bit confusing, but given the nice diff on the source string, they can piece together what happened.

7. Translators update the translations.

8. Weblate commits the translation changes, but not the changes to the comments:

    # user alert when frobber happened. 
   -the_alert=Frobber ist passiert!
   +the_alert=Flibber ist passiert!
   

9. The comment in the translation file is now out of sync (and translators will always see the concatenated comment).

Note: It is currently not clear to me, why the target file does not end up with the concatenated comment. This is what I'd expect based on the snippet referenced above. I will try and investigate this.

I'm happy to spend some time on this if you feel I can make a valuable contribution (all the possible solutions you have outlined would work for us). From the Java properties POV, I do not see the value of copying the comments, but I realize that this is only a single format :shrug:

I'll start by investigating the above note and hopefully I'll learn more.

In any case: thank you for this great software!

[nijel]

I think stripping comments while adding translated strings is the best approach. It is also the approach which needs least work ;-).

[gzm0]

TY for the quick feedback. I'll take a stab at this.

What I've found out so far RE not propagating comments: It seems the snippet above is from the wrapped ttkit unit. So it's not the one that is in the store.

My plan is to identify the point where a new unit gets created (when a translation is added) and attempt to prevent / remove comment propagation.

[nijel]

This is code you need to change:

https://github.com/WeblateOrg/weblate/blob/4c22536d9d14702193eb05ba7bba1297ddb623d2/weblate/formats/base.py#L274-L276

Most likely in TTKitFormat subclass and stripping comments there after the unit is cloned.