Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition

Introduction

Given a CTC-based trained ASR model, we proposed Single-Utterance Test-time Adaptation (SUTA) framework, which can adapt the source ASR model for one utterance by unsupervised objectives (such as entropy minimization, minimum class confusion). For details of SUTA's method and experimental results, please see our paper [link].

Our proposed SUTA has below advantages:

• Efficient adaptation for one single utterance
• Don't need to access the source data
• About 0.1s per 1s utterance for 10-steps adaptation

Installation

pip install -r requirements.txt

Data Preparation

Currently, our code only supports Librispeech/CHiME-3/Common voice En/TED-LIUM 3 You have to download datasets by your own.

Usage

The source ASR model is w2v2-base fine-tuned on Librispeech 960 hours. The pre-trained model is imported by Huggingface.

Run SUTA on different datasets:

bash scripts/{dataset_name: LS/CH/CV/TD}.sh

Results

TODO

• Support auto-regressive model
• More speech processing tasks beyond speech recognition

Contact

• Guan-Ting Lin [email] [email protected]

Citation

@article{lin2022listen,
  title={Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition},
  author={Lin, Guan-Ting and Li, Shang-Wen and Lee, Hung-yi},
  journal={arXiv preprint arXiv:2203.14222},
  year={2022}
}