firefox-translations-training
firefox-translations-training copied to clipboard
`download_mono` fails to load custom datasets with `/` in name
To set a custom-corpus
, I do something like this, which works at loading and saving the data:
custom-corpus_/custom_corpus/fingerspelling/devtest
However, doing something similar for mono-corpus
:
custom-mono_/custom_corpus/common_words/mono
It can not create the file, because it is creating this directory:
/training/data/spoken-signed/spoken_to_signed/original/mono/custom-mono_/custom_corpus/common_words/original/
but trying to save the file in this path
/training/data/spoken-signed/spoken_to_signed/original/mono/custom-mono_/custom_corpus/common_words/original/custom-mono_/custom_corpus/common_words/mono.original.spoken.gz
https://github.com/mozilla/firefox-translations-training/blob/main/pipeline/data/download-mono.sh#L28
Current workaround: manually create the directory it is trying to save to