weblate [XLIFF] Groups aren't respected or maintained during translation

Describe the bug XLIFF spec allows for a <group> element to logically organize trans-units into groups. There are ignored on import, and aren't preserved when generating the translation output.

To Reproduce A monolingual source file such as the following, Base.g.xlf

<?xml version="1.0" encoding="utf-8"?>
<xliff version="1.2" xmlns="urn:oasis:names:tc:xliff:document:1.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:oasis:names:tc:xliff:document:1.2 xliff-core-1.2-transitional.xsd">
  <file original="Base" datatype="plaintext" source-language="en_US">
    <body>
      <group id="body">
        <trans-unit id="135956960462609535">
          <source>Color</source>
        </trans-unit>
      </group>
    </body>
  </file>
</xliff>

Results in a translation output file en-GB.xlf

<?xml version='1.0' encoding='UTF-8'?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.1" version="1.1">
  <file original="NoName" source-language="en" datatype="plaintext" target-language="en_GB">
    <body>
      <trans-unit xml:space="preserve" approved="yes" id="Base///135956960462609535">
        <source></source>
        <target state="translated">Colour</target>
      </trans-unit>
    </body>
  </file>
</xliff>

Expected behavior The <group>s should be preserved between input and output files.

Server configuration and status

 * Weblate 3.3
 * Python 3.6.7
 * Django 2.1.3
 * Celery 4.2.1
 * celery-batches 0.2
 * six 1.11.0
 * social-auth-core 2.0.0
 * social-auth-app-django 3.1.0
 * django-appconf 1.0.2
 * translate-toolkit 2.3.1
 * Whoosh 2.7.4
 * defusedxml 0.5.0
 * Git 2.17.1
 * Pillow 5.1.0
 * python-dateutil 2.7.5
 * lxml 4.2.1
 * django-crispy-forms 1.7.2
 * django_compressor 2.2
 * djangorestframework 3.9.0
 * user-agents 1.1.0
 * jellyfish 0.6.1
 * pytz 2018.7
 * pyuca 1.2
 * PyYAML 3.12
 * tesserocr 2.3.1
 * Mercurial 4.5.3
 * git-svn 2.17.1
 * Database backends: django.db.backends.postgresql
 * Cache backends: avatar:FileBasedCache, default:RedisCache
 * Celery: redis://cache:6379/1, redis://cache:6379/1, regular
 * Platform: Linux 4.15.0-1035-azure (x86_64)

Additional context Groups are present in Spec 1.1.

Downloading the "original translation file" results in the following. Note the trans-unit has been moved outside the Group parent?

<?xml version='1.0' encoding='UTF-8'?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2" xsi:schemaLocation="urn:oasis:names:tc:xliff:document:1.2 xliff-core-1.2-transitional.xsd">
  <file original="Base" datatype="plaintext" source-language="en_US" target-language="en_GB">
    <body>
      <group id="body">
      </group>
    <trans-unit id="135956960462609535" approved="no">
          <source>Color</source>
        <target state="translated">Colour</target></trans-unit>
      </body>
  </file>
</xliff>

Jan 09 '19 03:01 XVII

Indeed Weblate completely ignores <group> in Xliff right now.

Jan 09 '19 20:01 nijel

Thanks for confirming.

We have to manually edit our files by hand after each translation commit which is a bit cumbersome. Hoping this can be added at some stage.

Jan 09 '19 22:01 XVII

In the interim, I think I might use an add-on to help inject this data in? It's just one group for us that must be present for our translation files to work.

I'm thinking EVENT_PRE_COMMIT ? I know it's a bit hard-codey, but as it applies to all our projects it might be a good stop-gap until it's implemented.

Do you think that'd work?

Jan 11 '19 00:01 XVII

I think that Weblate preserves order of the strings, so that could be a workaround. Just make sure that the grouped items are ordered next to each other.

Jan 11 '19 11:01 eighthave

In our scenario it's just one group across the board "body", so that should be easier to add at the start and end as a ham-fisted workaround.

What did you mean by making sure "the grouped items are ordered next to each other"?

Jan 13 '19 10:01 XVII

like sequentially in the file, one after the other

Jan 14 '19 14:01 eighthave

Hello, what is the current status here? It's been 4 years since the last message.

Mar 21 '23 06:03 pprossi

Still no progress here? Can someone give a hint on where we need to change things to make this work? I'am willing to contribute a PR (but need some guidance).

Currently we use the "Replace" Method to add new strings by the API but would like to use the "Add" Method.

I guess i need to modify add_unit https://github.com/WeblateOrg/weblate/blob/6685239fac2f052c5b543a7f1b0d94973a99c8df/weblate/trans/models/translation.py#L1359C7-L1359C7

And additionally the id is changed when using add_unit and prefixed with the component which is not what we want, the id should be preserved like it is when using the "Add" Method

Jan 10 '24 08:01 marknitek

I think this needs to be addressed in the translate-toolkit so that added unit would retain placement as in the original file.

Jan 13 '24 07:01 nijel

weblate weblate copied to clipboard

[XLIFF] Groups aren't respected or maintained during translation

weblate
weblate copied to clipboard