fairseq2
fairseq2 copied to clipboard
facebookresearch
FAIR Sequence Modeling Toolkit 2
Introduce support for StreamingLLM (https://arxiv.org/abs/2309.17453) for model exploration tasks.
**Describe the bug:** While the `AudioDecoder` can read a stereo wav file, the output of `WaveformToFbankConverter` is always `num_frames x num_mel_bins`. **Describe how to reproduce:** ```python audio_wav_path, device, dtype =...
Start using iwyu (include-what-you-use) in libfairseq2.
In libfairseq2 we have pretty good error reporting, but the same can't be said for logging. Introduce logging in C++ and figure out a way to integrate it with Python...
Right now we can't run our tests on GPU machines in GitHub Actions since there is still no support for them. Update our workflow files once the support is available.
Introduce support for sequence parallelism.
Port the architectural changes we introduced to w2v-BERT in seamless_communication to fairseq2 for this year's representation learning tasks. Also make sure that we not only support eval, but pretraining as...
The current implementation of `GumbelVectorQuantizer` is pretty much a copy/paste of its fairseq version. Brush it up and ensure we don't have any subtle issues in its implementation.
We still use a pretty naive Python implementation for RMSNorm. Migrate to a fused implementation before we start working on the LLM tasks.
The current LoRA API is not feature complete per our internal design doc. In particular, loading and saving LoRA adapters is not supported at the moment. Revise the current API...