Abdul Waheed

How come vanilla finetuning of bert with ~ 100x more trainable parameters compared to bert + adapter takes just 2x time?

2

## Environment info - `adapter-transformers` version: 3.0.1 - Platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic - Python version: 3.7.13 - PyTorch version (GPU?): 1.11.0+cu113 (True) - Tensorflow version (GPU?): 2.8.2 (True) - Flax version (CPU?/GPU?/TPU?):...

Training with unfreezed RoBERTa

8

Can someone train it with unfrozen RoBERTa and upload checkpoint?

Training stuck after loading the model?

6

Issue: Training doesn't begin after loading the model. **DS_REPORT** ``` (base) ext_abdul.waheed@p4-r69-a:~$ nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2020 NVIDIA Corporation Built on Wed_Jul_22_19:09:09_PDT_2020 Cuda compilation...

Multi-gpu training still has issues

8

**With int8** Error: RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cuda:1 **Without int8** RuntimeError: Expected all tensors to...

I am trying to train models on multiple nodes with DeepSpeed. Any resource for that? Seems like this PR #2404 was merged into the main but can't find any documentation...

ModuleNotFoundError with Multi-node training using SLURM

3

I am trying to train models on multiple nodes with SLURM as a workload manager. The Issue seems to be with the Python virtual environment not available to all nodes....

No Backend found error despite ffmpeg

3

**Versions**: ``` ffmpeg==4.3.2 librosa==0.9.2 audioread==3.0.0 ``` OS: `Gentoo` @sampsyo can you please take a look?

WER Filtering takes too long?

3

Hi @sanchit-gandhi ! Currently, WER filtering takes way too long with 8 workers, and going beyond 8 gives `self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory`. Also, it doesn't...

Forcing decoder to decode Arabic script only?

1

Hi @zaidalyafeai Why don't we force the decoder to decode only Arabic script during the evaluation? [Here](https://github.com/ARBML/whisperar/blob/63769eeaed3c25a2bea2c44fca3c217a8397113f/run_speech_recognition_seq2seq.py#L567)

### Anthology ID 2025.naacl-long.296 ### Type of Change Revision ### PDF of the Revision or Erratum [PaperID_648_uDistilWhisper.pdf](https://github.com/user-attachments/files/20217824/PaperID_648_uDistilWhisper.pdf) ### Brief Description of Changes The authors corrected minor inconsistencies in the reported...