Greg Tatum

I'm looking a bit into CI speeds, and I wanted to try to slim down the model even more.

It takes 15 minutes to run one test, I think due to the install for the virtual environment that contains cuda and pytorch dependencies. tests/test_data_importer.py::test_basic_corpus_import[mtdata-Neulab-tedtalks_test-1-eng-rus] PASSED [ 29%] https://share.firefox.dev/3ZclPBD

* [Profile of the CI tasks](https://share.firefox.dev/47gHESv) * [evaluate-backward-flores-devtest-ru-en](https://share.firefox.dev/4gfdgvW) | task | runtime | | ---- | ------- | | evaluate-backward-flores-devtest-ru-en | 16m | | evaluate-teacher-ensemble-flores-devtest-ru-en | 18m | nvidia cudnn...

The regexes for characters in a language match on only 1 character, not the entire token

4

https://github.com/hplt-project/OpusCleaner/blob/main/opuscleaner/filters/clean_common.py The regexes for characters in a language match on only 1 character, not the entire token: ```py 'ca': r'[a-zÀàÈèÉéÍíÒòÓóÚúÇç]', 'cs': r'[a-zÁáČčĎďÉéěÍíŇňÓóŘřŠšŤťÚúůÝýŽž]', 'da': r'[a-zÆæØøÅå]', 'de': r'[a-zÄäÖöÜüß]', ``` As opposed to:...

https://opus.nlpl.eu/sample/bn&en/pmindia&v1/sample The order should be swapped

formatBytes is ambiguous as it displays mebibyte (MiB) numbers but the label suggests megabytes (MB)

2

I had to look up the source code when comparing memory usage across tools while doing a memory analysis to confirm that the profiler was showing mebibytes, and not actually...

Remove "tracing" marker type, and migrate to the "category" field.

3

Right now the markers of type "tracing" don't work too nicely with the new marker schema. "type" should be the differentiator on how we display things. However, these markers use...

Port over the minimal code for the Task Profiler implementation

2

This is the initial code needed for converting Taskcluster tasks and live logs into the Firefox Profiler format. It is taken from: https://github.com/gregtatum/taskcluster-tools The goal with this PR is not...

Add support for viewing tasks and taskgroups in the profiler

3

While working on [Firefox Translations](https://github.com/mozilla/translations), I have been building some experimental [Taskcluster Tools](https://gregtatum.github.io/taskcluster-tools/) for use with training. https://github.com/gregtatum/taskcluster-tools One of the tools that would probably be useful to a general...