FunASR

FunASR copied to clipboard

After I change the batch_size to 15000 for training, I have a size mismatch problem in the decoding phase, please answer. Below are the specific error codes: 2023-12-04 09:45:29,392 - modelscope - INFO - PyTorch version 1.12.0+cu116 Found. 2023-12-04 09:45:29,392 - modelscope - INFO - Loading ast index from /home/weiyangjie/.cache/modelscope/ast_indexer 2023-12-04 09:45:29,412 - modelscope - INFO - Loading done! Current index file version is 1.9.5, with md5 3bc417c7d54214f5b7696ccb26e59cb8 and a total number of 945 components indexed /home/weiyangjie/anaconda3/envs/funasr/bin/python /newdisk0630/wbb/Project/FunASR/funasr/bin/asr_inference_launch.py --batch_size 2 --ngpu 1 --njob 1 --gpuid_list 1 --data_path_and_name_and_type ../DATA/data/dev/wav.scp,speech,sound --cmvn_file ../DATA/data/train/cmvn/am.mvn --key_file ./exp/baseline_train_asr_paraformer_conformer_12e_6d_2048_256_zh_char_exp1/decode_asr_transformer_noctc_1best/valid.acc.ave_10best.pb/dev/logdir/keys.1.scp --asr_train_config ./exp/baseline_train_asr_paraformer_conformer_12e_6d_2048_256_zh_char_exp1/config.yaml --asr_model_file ./exp/baseline_train_asr_paraformer_conformer_12e_6d_2048_256_zh_char_exp1/valid.acc.ave_10best.pb --output_dir ./exp/baseline_train_asr_paraformer_conformer_12e_6d_2048_256_zh_char_exp1/decode_asr_transformer_noctc_1best/valid.acc.ave_10best.pb/dev/logdir/output.1 --mode paraformer --config conf/decode_asr_transformer_noctc_1best.yaml 2023-12-04 09:45:29,488 (asr_inference_launch:2249) INFO: Decoding args: {'log_level': 'INFO', 'output_dir': './exp/baseline_train_asr_paraformer_conformer_12e_6d_2048_256_zh_char_exp1/decode_asr_transformer_noctc_1best/valid.acc.ave_10best.pb/dev/logdir/output.1', 'ngpu': 1, 'njob': 1, 'gpuid_list': '1', 'seed': 0, 'dtype': 'float32', 'num_workers': 1, 'data_path_and_name_and_type': [('../DATA/data/dev/wav.scp', 'speech', 'sound')], 'key_file': './exp/baseline_train_asr_paraformer_conformer_12e_6d_2048_256_zh_char_exp1/decode_asr_transformer_noctc_1best/valid.acc.ave_10best.pb/dev/logdir/keys.1.scp', 'hotword': None, 'allow_variable_data_keys': False, 'mc': False, 'vad_infer_config': None, 'vad_model_file': None, 'punc_infer_config': None, 'punc_model_file': None, 'cmvn_file': '../DATA/data/train/cmvn/am.mvn', 'asr_train_config': './exp/baseline_train_asr_paraformer_conformer_12e_6d_2048_256_zh_char_exp1/config.yaml', 'asr_model_file': './exp/baseline_train_asr_paraformer_conformer_12e_6d_2048_256_zh_char_exp1/valid.acc.ave_10best.pb', 'sv_model_file': None, 'lm_train_config': None, 'lm_file': None, 'word_lm_train_config': None, 'word_lm_file': None, 'ngram_file': None, 'model_tag': None, 'beam_search_config': {}, 'batch_size': 2, 'nbest': 5, 'beam_size': 1, 'penalty': 0.0, 'maxlenratio': 0.0, 'minlenratio': 0.0, 'ctc_weight': 0.0, 'lm_weight': 0.15, 'ngram_weight': 0.9, 'streaming': False, 'fake_streaming': False, 'full_utt': False, 'chunk_size': 16, 'left_context': 16, 'right_context': 0, 'display_partial_hypotheses': False, 'quantize_asr_model': False, 'quantize_modules': None, 'quantize_dtype': 'qint8', 'token_type': None, 'bpemodel': None, 'token_num_relax': 1, 'decoding_ind': 0, 'decoding_mode': 'model1', 'ctc_weight2': 0.0, 'mode': 'paraformer'} 2023-12-04 09:45:29,663 (build_asr_model:283) INFO: Vocabulary size: 4234 2023-12-04 09:45:29,664 (conformer_encoder:560) WARNING: Using legacy_rel_pos and it will be deprecated in the future. 2023-12-04 09:45:30,564 (conformer_encoder:667) WARNING: Using legacy_rel_selfattn and it will be deprecated in the future. 2023-12-04 09:45:30,889 (build_model_from_file:56) INFO: model_file is ./exp/baseline_train_asr_paraformer_conformer_12e_6d_2048_256_zh_char_exp1/valid.acc.ave_10best.pb Traceback (most recent call last): File "/home/weiyangjie/anaconda3/envs/funasr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/weiyangjie/anaconda3/envs/funasr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/newdisk0630/wbb/Project/FunASR/funasr/bin/asr_inference_launch.py", line 2263, in main() File "/newdisk0630/wbb/Project/FunASR/funasr/bin/asr_inference_launch.py", line 2258, in main inference_pipeline = inference_launch(**kwargs) File "/newdisk0630/wbb/Project/FunASR/funasr/bin/asr_inference_launch.py", line 2207, in inference_launch return inference_paraformer(**kwargs) File "/newdisk0630/wbb/Project/FunASR/funasr/bin/asr_inference_launch.py", line 310, in inference_paraformer speech2text = Speech2TextParaformer(**speech2text_kwargs) File "/newdisk0630/wbb/Project/FunASR/funasr/bin/asr_infer.py", line 290, in init asr_model, asr_train_args = build_model_from_file( File "/newdisk0630/wbb/Project/FunASR/funasr/build_utils/build_model_from_file.py", line 80, in build_model_from_file model.load_state_dict(model_dict, strict=False) File "/home/weiyangjie/anaconda3/envs/funasr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1604, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for Paraformer: size mismatch for encoder.embed.conv.0.weight: copying a param with shape torch.Size([256, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 256, 3, 3]).