firefox-translations-training icon indicating copy to clipboard operation
firefox-translations-training copied to clipboard

`download_mono` fails to load custom datasets with `/` in name

Open AmitMY opened this issue 2 years ago • 0 comments

To set a custom-corpus, I do something like this, which works at loading and saving the data:

custom-corpus_/custom_corpus/fingerspelling/devtest

However, doing something similar for mono-corpus:

custom-mono_/custom_corpus/common_words/mono

It can not create the file, because it is creating this directory:

/training/data/spoken-signed/spoken_to_signed/original/mono/custom-mono_/custom_corpus/common_words/original/

but trying to save the file in this path

/training/data/spoken-signed/spoken_to_signed/original/mono/custom-mono_/custom_corpus/common_words/original/custom-mono_/custom_corpus/common_words/mono.original.spoken.gz

https://github.com/mozilla/firefox-translations-training/blob/main/pipeline/data/download-mono.sh#L28


Current workaround: manually create the directory it is trying to save to

AmitMY avatar Aug 09 '22 18:08 AmitMY