firefox-translations-training
firefox-translations-training copied to clipboard
mozilla
Training pipelines for Firefox Translations neural machine translation models
https://firefox-ci-tc.services.mozilla.com/tasks/IujzQFOHSyarfuGZTqYOHg/runs/0/logs/public/logs/live.log ``` [task 2024-06-17T20:26:28.119Z] Decompress: 7/20 files. Current: ...file.16.out.zst : 17 MB... [task 2024-06-17T20:26:28.285Z] Decompress: 7/20 files. Current: ...file.16.out.zst : 71 MB... [task 2024-06-17T20:26:28.452Z] Decompress: 7/20 files. Current: ...file.16.out.zst :...
I'm trying to continue the pipeline after we get the results of the `evaluate-teacher-ensemble` task. By doing something like: ``` target-stage: all start-stage: translate-corpus previous_group_ids: ["N1O85rIASLmCwKfUpCvTlw"] ``` or ``` start-stage:...
https://firefox-ci-tc.services.mozilla.com/tasks/IVGFh-gRSaOmOGjkaJGqsw/runs/0/logs/public/logs/live.log ``` [task 2024-05-24T17:10:46.027Z] 2024-05-24 17:07:38.234457: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA...
While working on the big training and bug fixes I ran into many issues with scheduling specific tasks. Basically, the graph and caches can be in an arbitrary state and...
Example runs: https://firefox-ci-tc.services.mozilla.com/tasks/groups/b8_PxNNiSKWyuugT0V8faA https://firefox-ci-tc.services.mozilla.com/tasks/groups/BFgoyXuvRHe0kK4J9d-Lgw https://firefox-ci-tc.services.mozilla.com/tasks/groups/LphXRhZoScGYSNAPKCiVfQ See #629 It might be related to #549
Closes #408 Closes #668 Closes #610 Still have to test with online publication.
An example is here (id-en), where the task timed out. https://firefox-ci-tc.services.mozilla.com/tasks/NWpySVWkT2SVBY3AOkDaIQ
I have an issue where the Finnish -> English translation of Firefox alters numbers. Reproduction: 1. Translate `283,82 €` from Finnish to English. Expected outcome: `283,82 €` or some variation...
Bosnian-Croatian-Montenegrin-Serbian are a language group that are very similar, and mutually intelligible. Per [Wikipedia](https://en.wikipedia.org/wiki/Serbo-Croatian): > It is a [pluricentric language](https://en.wikipedia.org/wiki/Pluricentric_language) with four[[16]](https://en.wikipedia.org/wiki/Serbo-Croatian#cite_note-16) [mutually intelligible](https://en.wikipedia.org/wiki/Mutually_intelligible) [standard varieties](https://en.wikipedia.org/wiki/Standard_language), namely [Serbian](https://en.wikipedia.org/wiki/Serbian_language), [Croatian](https://en.wikipedia.org/wiki/Croatian_language), [Bosnian](https://en.wikipedia.org/wiki/Bosnian_language),...
