Xingchen Song(宋星辰)

涉及到修改的可不可以通过下面的方式： ```py from transformers import XXXModelForCasulLM from wenet.transformer.asr_model import AsrModel class NewModel(nn.Module, AsrModel, XXXModelForCasulLM): def __init__(self, ...): # init father super().__init__() # add new member if needed, i.e., self.new_member =...

## The Dataset of Speech Recognition (ASR) / Speech Translation (ST) ### **Chinese** | name | duration/h | address | remark | | --- | --- | --- | ---...

## The Dataset of Speech Synthesis **Chinese** | name | duration/h | address | remark | --- | --- | --- | --- | Aishell3 | 85 | https://openslr.org/93/ |...

## The Dataset of Speech Recognition & Speaker Diarization **Chinese** | name | duration/h | address | remark | application | --- | --- | --- | --- | ---...

## The Dataset of Speaker Recognition **Chinese** | name | duration/h | address | remark | application | --- | --- | --- | --- | --- | CN-Celeb |...

## The Resource of Crawler | name | type | address | remark | application | --- | --- | --- | --- | --- | voicetube | video |...

可以参考train_utils.py::wenetjoin函数，目前是只对deepspeed打开了，ddp没开，可以把他打开

建议直接用deepspeed吧，小模型会快10%，大模型ddp直接没法训，deepspeed配置可参考examples/aishell/whisper/conf/dsstage1.json

> ok, 回头试下，我理解wenet_join的作用和这里[espnet](https://github.com/espnet/espnet/blob/7c140c2ac9b4f642acb36131217dd984d4601681/espnet2/train/trainer.py#L539)是一样的吧。 功能类似，espnet的实现比较裸

> > 建议直接用deepspeed吧，小模型会快10%，大模型ddp直接没法训，deepspeed配置可参考examples/aishell/whisper/conf/dsstage1.json > > 是这样的，我用deep speed训的icefall中的zip former，但是卡在了loss.backward（），训conformer是正常的，所以我一直就用的DDP 贡献下zipformer实现，我给你调通deepspeed😁