firefox-translations-training

firefox-translations-training copied to clipboard

A job does not exit on OpusCleaner error

1

See https://firefox-ci-tc.services.mozilla.com/tasks/B5S8nc1OTI6hOxCkGDgG9A/runs/0/logs/public/logs/live.log

eu9ene

In the cleaning task, output statistics about filtered out/kept sentences and maybe attach list of filtered out sentences as an artifact

4

This might help debugging cleaning.

marco-c

As a follow-up for the [initial integration](https://github.com/mozilla/firefox-translations-training/pull/163) we should: - Use the OpusCleaner UI and produce the full training config for all datasets for a language pair - Figure out...

eu9ene

Snakemake dry run is broken

1

``` [task 2024-01-12T19:21:59.577Z] Start the dry run [task 2024-01-12T19:21:59.578Z] echo "Dry run with config configs/config.prod.yml and profile local" [task 2024-01-12T19:21:59.578Z] source /builds/worker/artifacts/mambaforge/etc/profile.d/conda.sh ; conda activate ; conda activate snakemake [task...

eu9ene

Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.0.5 to 2.0.7. Release notes Sourced from urllib3's releases. 2.0.7 Made body stripped from HTTP requests changing the request method to GET after HTTP 303 "See Other"...

dependabot[bot]

Publish experiment metrics separetely from Marian training logs

1

Publishing to W&B using the last step (instead of step 0). Part of #164 \ Refs. https://github.com/mozilla/firefox-translations-training/pull/309#discussion_r1439963563

vrigal

Splitting main to multiple functions. Part of #164 \ Refs. https://github.com/mozilla/firefox-translations-training/pull/309#discussion_r1439933153

vrigal

It would be interesting to display stats about W&B publication failures Part of #164 \ Refs. https://github.com/mozilla/firefox-translations-training/pull/309#discussion_r1439967721

vrigal

Removing the old `Publisher.publish` method. Parsers that do things at the end may implement custom behavior during the `close` call (e.g. writing a file from cached results). Part of #164...

vrigal

`merge-corpus` and `merge-mono` do not work no cleaning is done

2

In this training run I didn't apply any cleaning configuration, which broke the task building process: https://firefox-ci-tc.services.mozilla.com/tasks/bb_He8xySyee_GnvB4Ucog Here is the config that I was using: https://github.com/mozilla/firefox-translations-training/blob/train-en-ca/configs/tc.prod.yml

gregtatum