firefox-translations-training
firefox-translations-training copied to clipboard
mozilla
Training pipelines for Firefox Translations neural machine translation models
Retrain en-ru teacher with the new custom configs. Other things to try: - one stage training - updated bicleaner-ai [skip ci]
From https://firefox-ci-tc.services.mozilla.com/tasks/CcBRvVfDT229U-iLBThC5w/runs/2: ``` [task 2024-05-02T00:40:40.350Z] [tracking ERROR] Publication failed: Invalid config section: while scanning a simple key [task 2024-05-02T00:40:40.350Z] in "", line 493, column 1: [task 2024-05-02T00:40:40.350Z] Loaded model has...
New Taskcluster error: https://firefox-ci-tc.services.mozilla.com/tasks/aJKhHjeGSzmLPU7p9V8sLg/runs/10/logs/public/logs/live.log
Aside from the included tests, I tested this by hand. What I did was: * Start a `train-backwards` job: https://firefox-ci-tc.services.mozilla.com/tasks/bNP6s4FaRwaU7Bz8gxgJ-Q/runs/0 * Wait for it to checkpoint * Simulate a spot...
Using some sort of “fuzzing”/”genetic algorithm” to choose the rules (where the oracle is a LLM). Related paper: Published as a conference paper at ICLR 2024 [CONNECTING LARGE LANGUAGE MODELS...
https://github.com/mozilla/firefox-translations-training/blob/a7735635b5031ab016b8ae3f2a803e88119d65bb/docs/img/DAG.svg shows `collect` coming right after `split`, which I'm pretty sure is wrong...even for Snakemake. Here's the previous version of the DAG for comparison: https://raw.githubusercontent.com/mozilla/firefox-translations-training/ff35476e16985c3cc9e21e50356a4c90efb623d0/docs/img/DAG.svg
For example: https://firefox-ci-tc.services.mozilla.com/tasks/IvbeCQBuRuKIOaeOIGEfHg We had an initial run of this task which got spot killed. Subsequent runs failed with: ``` error: cache /builds/worker/checkouts is not empty and is missing a...
There are scenarios where we either have no thresholds to override, or are disabling bicleaning altogether. We should update [the action schema](https://github.com/mozilla/firefox-translations-training/blob/a7735635b5031ab016b8ae3f2a803e88119d65bb/taskcluster/translations_taskgraph/actions/train.py#L141) to allow for this. (Note: using an empty...