icefall
icefall copied to clipboard
k2-fsa
Major Difference in the output of Torch Model and Onnx Model for Zipformer2
Hi,
I have trained zipformer2 (without streaming) model with my dataset.
Training command: ./zipformer/train.py --num-epochs 40 --start-epoch 1 --use-fp16 1 --enable-musan False --exp-dir zipformer/exp-small --causal 0 --num-encoder-layers 2,2,2,2,2,2 --feedforward-dim 512,768,768,768,768,768 --encoder-dim 192,256,256,256,256,256 --encoder-unmasked-dim 192,192,192,192,192,192 --base-lr 0.04 --full-libri 1
When I decode from the .pt model, the output is nearly same as the GT
Decoding Command (for .pt Model): ./zipformer/decode.py --epoch 40 --use-averaged-model False --avg 1 --exp-dir zipformer/exp-small/ --decoding-method greedy_search --causal 0 --num-encoder-layers 2,2,2,2,2,2 --feedforward-dim 512,768,768,768,768,768 --encoder-dim 192,256,256,256,256,256 --encoder-unmasked-dim 192,192,192,192,192,192 --chunk-size 16 --left-context-frames 64 --bpe-model data/lang_bpe_500/bpe.model --manifest-dir data/test/fbank/
But when I convert the model in onnx and decode using the onnx model, most of the segments are missing in the decoded output.
Onnx Conversion Commnad: ./zipformer/export-onnx-bhaswati.py --tokens data/lang_bpe_500/tokens.txt --use-averaged-model 1 --epoch 40 --avg 1 --exp-dir zipformer/exp-small/ --num-encoder-layers "2,2,2,2,2,2" --feedforward-dim "512,768,768,768,768,768" --encoder-dim "192,256,256,256,256,256" --encoder-unmasked-dim "192,192,192,192,192,192" --causal False --chunk-size 16 --left-context-frames 64
Decoding Command (for .onnx Model): ./zipformer/onnx_pretrained.py --encoder-model-filename zipformer/exp-small/encoder-epoch-40-avg-1.int8.onnx --decoder-model-filename zipformer/exp-small/decoder-epoch-40-avg-1.int8.onnx --joiner-model-filename [zipformer/exp-small/joiner-epoch-40-avg-1.int8.onnx --tokens data/lang_bpe_500/tokens.txt audio.wav
Decoding Command (for .pt Model): ./zipformer/decode.py --epoch 40 --use-averaged-model False
Onnx Conversion Commnad: ./zipformer/export-onnx-bhaswati.py --tokens data/lang_bpe_500/tokens.txt --use-averaged-model 1
Could you use the same value for --use-averaged-model 1 and then test it again?
most of the segments are missing in the decoded output.
Could you give a detailed example out?
I have uploaded the model and the audio here, if it helps.
Below are the two outputs from pth model and onnx model.
Onnx Conversion Commnad: ./zipformer/export-onnx-bhaswati.py --tokens data/lang_bpe_500/tokens.txt --use-averaged-model 1 --epoch 40 --avg 1 --exp-dir zipformer/exp-small/ --num-encoder-layers "2,2,2,2,2,2" --feedforward-dim "512,768,768,768,768,768" --encoder-dim "192,256,256,256,256,256" --encoder-unmasked-dim "192,192,192,192,192,192" --causal False --chunk-size 16 --left-context-frames 64
Decoding Command (for .onnx Model): ./zipformer/onnx_pretrained.py --encoder-model-filename zipformer/exp-small/encoder-epoch-40-avg-1.int8.onnx --decoder-model-filename zipformer/exp-small/decoder-epoch-40-avg-1.int8.onnx --joiner-model-filename [zipformer/exp-small/joiner-epoch-40-avg-1.int8.onnx --tokens data/lang_bpe_500/tokens.txt audio.wav
Seems like there is some issue while converting the model to onnx. Majority of the portion of the audio is missing in onnx decode result.
I have uploaded the model and the audio here, if it helps.
Could you post some text results? The files in your link are > 100MB, which is too large for me to download.
Also, could you test not using .int8.onnx?
Below I am attaching the output of a particular audio from .pth, int8.onnx and .onnx model. The output is blank for .onnx model
pth: θBΩə θF μFOCSQə βMλə OBλə C∞ψθLλJ∞ζə ζəλə θBΩə BC ΩK@ə μFOCSQə C∞βJO!ə ΩK@ə CSOφBθəλ@ə βθF @Jρə γKOə L∞əμC K∞@ə BC βλMεə θLλə J∞@ə !F OMμə εC γKOə γə!ə ψC @C@ə∞!ə !əλ∞ə Mβə ζMλC Kεə BC M@Cγəμə ζəλə ΩKμL θBΩə OK∞ə θF ΩCθəλə εC
int8.onnx: βMλə C∞ψθLλJ∞ζə θBΩə ΩK@ə C∞βə ə
onnx: ""
Could you also post the logs of export-onnx-bhaswati.py and decode.py? Only the first part of the logs that contains detailed parameters is enough.
Logs of export-onnx.py:
2023-11-23 15:35:49,088 INFO [export-onnx-bhaswati.py:434] device: cuda:0 2023-11-23 15:35:49,091 INFO [export-onnx-bhaswati.py:440] {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 50, 'reset_interval': 200, 'valid_interval': 3000, 'feature_dim': 80, 'subsampling_factor': 4, 'warm_step': 2000, 'env_info': {'k2-version': '1.24.3', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': '7ff6d891905ff482364f2d0015867b00d89dd8c7', 'k2-git-date': 'Fri Jun 16 12:10:37 2023', 'lhotse-version': '1.16.0.dev+git.aa073f6a.clean', 'torch-version': '1.13.1+cu117', 'torch-cuda-available': True, 'torch-cuda-version': '11.7', 'python-version': '3.9', 'icefall-git-branch': 'master', 'icefall-git-sha1': 'fc2df07-dirty', 'icefall-git-date': 'Wed Aug 16 20:02:41 2023', 'icefall-path': '/NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall', 'k2-path': '/home/sanjukta/anaconda3/envs/zipf1/lib/python3.9/site-packages/k2/init.py', 'lhotse-path': '/NAS1/sanjukta_repo_falcon1/zip_exp_6/lhotse/lhotse/init.py', 'hostname': 'asus-System-Product-Name', 'IP address': '127.0.1.1'}, 'epoch': 40, 'iter': 0, 'avg': 1, 'use_averaged_model': False, 'exp_dir': PosixPath('zipformer/exp-ignite_via/25_10_2023'), 'tokens': 'data/ignite_via/20_10_2023/lang_bpe_500/tokens.txt', 'context_size': 2, 'num_encoder_layers': '2,2,2,2,2,2', 'downsampling_factor': '1,2,4,8,4,2', 'feedforward_dim': '512,768,768,768,768,768', 'num_heads': '4,4,4,8,4,4', 'encoder_dim': '192,256,256,256,256,256', 'query_head_dim': '32', 'value_head_dim': '12', 'pos_head_dim': '4', 'pos_dim': 48, 'encoder_unmasked_dim': '192,192,192,192,192,192', 'cnn_module_kernel': '31,31,15,15,15,31', 'decoder_dim': 512, 'joiner_dim': 512, 'causal': False, 'chunk_size': '16', 'left_context_frames': '64', 'use_transducer': True, 'use_ctc': False, 'blank_id': 0, 'vocab_size': 500} 2023-11-23 15:35:49,091 INFO [export-onnx-bhaswati.py:442] About to create model 2023-11-23 15:35:51,104 INFO [checkpoint.py:112] Loading checkpoint from zipformer/exp-ignite_via/25_10_2023/epoch-40.pt 2023-11-23 15:35:51,351 INFO [export-onnx-bhaswati.py:546] encoder parameters: 22121363 2023-11-23 15:35:51,351 INFO [export-onnx-bhaswati.py:547] decoder parameters: 522752 2023-11-23 15:35:51,351 INFO [export-onnx-bhaswati.py:548] joiner parameters: 256500 2023-11-23 15:35:51,351 INFO [export-onnx-bhaswati.py:549] total parameters: 22900615 2023-11-23 15:35:51,351 INFO [export-onnx-bhaswati.py:560] Exporting encoder /NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall/egs/librispeech/ASR/zipformer/scaling.py:1431: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. zero = torch.tensor(0.0, dtype=x.dtype, device=x.device) /NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall/egs/librispeech/ASR/zipformer/scaling.py:1358: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. zero = torch.tensor(0.0, dtype=x.dtype, device=x.device) /NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall/egs/librispeech/ASR/zipformer/scaling.py:436: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert x.shape[self.channel_dim] == self.num_channels /NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall/egs/librispeech/ASR/zipformer/subsampling.py:334: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert x.size(1) == x_lens.max().item() , (x.size(1), x_lens.max()) /NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall/egs/librispeech/ASR/zipformer/subsampling.py:334: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert x.size(1) == x_lens.max().item() , (x.size(1), x_lens.max()) /NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall/icefall/utils.py:1261: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! max_len = max(max_len, lengths.max()) /NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall/egs/librispeech/ASR/zipformer/scaling.py:1596: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if num_channels <= x.shape[-1]: /NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall/egs/librispeech/ASR/zipformer/zipformer.py:1505: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert p.shape[-1] == num_heads * pos_head_dim /NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall/egs/librispeech/ASR/zipformer/zipformer.py:1580: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert attn_scores.shape == (num_heads, batch_size, seq_len, seq_len) /NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall/egs/librispeech/ASR/zipformer/zipformer.py:1591: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert key_padding_mask.shape == (batch_size, seq_len), key_padding_mask.shape /NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall/egs/librispeech/ASR/zipformer/zipformer.py:1968: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert attn_weights.shape == (num_heads, batch_size, seq_len, seq_len) /NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall/egs/librispeech/ASR/zipformer/zipformer.py:1777: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert attn_weights.shape == (num_heads, batch_size, seq_len, seq_len) /NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall/egs/librispeech/ASR/zipformer/zipformer.py:1237: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert src.shape[0] == d_seq_len * ds /home/sanjukta/anaconda3/envs/zipf1/lib/python3.9/site-packages/torch/nn/modules/module.py:1171: UserWarning: operator() profile_node %108 : int = prim::profile_ivalue(%106) does not have profile information (Triggered internally at ../torch/csrc/jit/codegen/cuda/graph_fuser.cpp:105.) return self.forward(*input, **kwargs) /home/sanjukta/anaconda3/envs/zipf1/lib/python3.9/site-packages/torch/onnx/utils.py:823: UserWarning: no signature found for <torch.ScriptMethod object at 0x7f117c77ce00>, skipping _decide_input_format warnings.warn(f"{e}, skipping _decide_input_format") 2023-11-23 15:37:11,493 INFO [export-onnx-bhaswati.py:324] meta_data: {'model_type': 'zipformer2', 'version': '1', 'model_author': 'k2-fsa', 'comment': 'non-streaming zipformer2'} 2023-11-23 15:37:13,434 INFO [export-onnx-bhaswati.py:567] Exported encoder to zipformer/exp-ignite_via/25_10_2023/encoder-epoch-40-avg-1.onnx 2023-11-23 15:37:13,434 INFO [export-onnx-bhaswati.py:569] Exporting decoder /home/sanjukta/anaconda3/envs/zipf1/lib/python3.9/site-packages/torch/onnx/_internal/jit_utils.py:258: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at ../torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.) _C._jit_pass_onnx_node_shape_type_inference(node, params_dict, opset_version) /home/sanjukta/anaconda3/envs/zipf1/lib/python3.9/site-packages/torch/onnx/utils.py:687: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at ../torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.) _C._jit_pass_onnx_graph_shape_type_inference( /home/sanjukta/anaconda3/envs/zipf1/lib/python3.9/site-packages/torch/onnx/utils.py:1178: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at ../torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.) _C._jit_pass_onnx_graph_shape_type_inference( 2023-11-23 15:37:13,556 INFO [export-onnx-bhaswati.py:576] Exported decoder to zipformer/exp-ignite_via/25_10_2023/decoder-epoch-40-avg-1.onnx 2023-11-23 15:37:13,556 INFO [export-onnx-bhaswati.py:578] Exporting joiner 2023-11-23 15:37:13,556 INFO [export-onnx-bhaswati.py:394] joiner dim: 512 2023-11-23 15:37:13,600 INFO [export-onnx-bhaswati.py:585] Exported joiner to zipformer/exp-ignite_via/25_10_2023/joiner-epoch-40-avg-1.onnx 2023-11-23 15:37:13,600 INFO [export-onnx-bhaswati.py:590] Generate int8 quantization models 2023-11-23 15:37:14.783658509 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/0/encoder_pos/Shape_5_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783711448 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/0/encoder_pos/Constant_46_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783717560 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/0/encoder_pos/Constant_24_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783723972 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/0/encoder_pos/Constant_37_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783731836 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '753'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783735503 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/0/encoder_pos/Constant_39_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783739170 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/0/encoder_pos/Shape_10_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783762003 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/0/encoder_pos/Constant_22_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783765860 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/0/encoder_pos/Shape_15_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783769677 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/0/encoder_pos/Constant_48_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783858203 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/1/encoder/encoder_pos/Shape_15_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783863673 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/1/encoder/encoder_pos/Shape_10_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783867200 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/1/encoder/encoder_pos/Constant_22_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783873812 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/1/encoder/encoder_pos/Constant_24_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783878230 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/1/encoder/encoder_pos/Shape_5_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783881827 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/1/encoder/encoder_pos/Constant_46_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783891014 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '2007'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783900301 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/1/encoder/encoder_pos/Constant_48_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783906613 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/1/encoder/encoder_pos/Constant_37_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.783910831 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/1/encoder/encoder_pos/Constant_39_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784029052 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/2/encoder/encoder_pos/Shape_10_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784036476 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/2/encoder/encoder_pos/Shape_5_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784041325 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/2/encoder/encoder_pos/Constant_48_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784046865 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/2/encoder/encoder_pos/Constant_24_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784050743 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/2/encoder/encoder_pos/Constant_37_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784067905 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '3296'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784072293 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/2/encoder/encoder_pos/Shape_15_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784076581 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/2/encoder/encoder_pos/Constant_22_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784080248 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/2/encoder/encoder_pos/Constant_39_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784085097 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/2/encoder/encoder_pos/Constant_46_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784169334 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/3/encoder/encoder_pos/Shape_15_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784173653 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/3/encoder/encoder_pos/Shape_10_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784178191 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/3/encoder/encoder_pos/Shape_5_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784183551 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/3/encoder/encoder_pos/Constant_48_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784186947 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/3/encoder/encoder_pos/Constant_46_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784192327 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/3/encoder/encoder_pos/Constant_24_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784195924 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/3/encoder/encoder_pos/Constant_22_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784212395 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '4585'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784217915 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/3/encoder/encoder_pos/Constant_39_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784223155 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/3/encoder/encoder_pos/Constant_37_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784303325 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/4/encoder/encoder_pos/Shape_15_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784308084 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/4/encoder/encoder_pos/Shape_5_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784316019 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/4/encoder/encoder_pos/Constant_48_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784330065 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '5869'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784336447 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/4/encoder/encoder_pos/Shape_10_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784343690 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/4/encoder/encoder_pos/Constant_22_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784347648 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/4/encoder/encoder_pos/Constant_37_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784351415 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/4/encoder/encoder_pos/Constant_46_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784355733 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/4/encoder/encoder_pos/Constant_39_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784359971 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/4/encoder/encoder_pos/Constant_24_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784434019 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/5/encoder/encoder_pos/Shape_10_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784441774 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/5/encoder/encoder_pos/Constant_48_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784445401 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/5/encoder/encoder_pos/Constant_46_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784451773 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/5/encoder/encoder_pos/Shape_5_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784459747 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '7151'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784463224 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/5/encoder/encoder_pos/Constant_22_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784467983 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/5/encoder/encoder_pos/Constant_24_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784472902 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/5/encoder/encoder_pos/Shape_15_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784482480 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/5/encoder/encoder_pos/Constant_37_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:14.784486307 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '/encoder/5/encoder/encoder_pos/Constant_39_output_0'. It is not used by any node and should be removed from the model. 2023-11-23 15:37:15,913 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder_embed/Reshape_output_0" not specified 2023-11-23 15:37:15,956 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/Slice_output_0" not specified 2023-11-23 15:37:16,063 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/encoder_pos/Unsqueeze_11_output_0" not specified 2023-11-23 15:37:16,079 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/feed_forward1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,089 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/Add_output_0" not specified 2023-11-23 15:37:16,101 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/nonlin_attention/Mul_4_output_0" not specified 2023-11-23 15:37:16,107 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/Add_1_output_0" not specified 2023-11-23 15:37:16,114 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/self_attn1/Reshape_1_output_0" not specified 2023-11-23 15:37:16,118 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/Add_2_output_0" not specified 2023-11-23 15:37:16,128 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/conv_module1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,135 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/Add_3_output_0" not specified 2023-11-23 15:37:16,147 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/feed_forward2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,159 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/bypass_mid/Add_output_0" not specified 2023-11-23 15:37:16,165 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/self_attn2/Reshape_1_output_0" not specified 2023-11-23 15:37:16,170 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/Add_5_output_0" not specified 2023-11-23 15:37:16,181 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/conv_module2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,188 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/Add_6_output_0" not specified 2023-11-23 15:37:16,202 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/feed_forward3/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,216 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.0/bypass/Add_output_0" not specified 2023-11-23 15:37:16,238 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/feed_forward1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,248 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/Add_output_0" not specified 2023-11-23 15:37:16,260 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/nonlin_attention/Mul_4_output_0" not specified 2023-11-23 15:37:16,267 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/Add_1_output_0" not specified 2023-11-23 15:37:16,273 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/self_attn1/Reshape_1_output_0" not specified 2023-11-23 15:37:16,278 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/Add_2_output_0" not specified 2023-11-23 15:37:16,288 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/conv_module1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,295 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/Add_3_output_0" not specified 2023-11-23 15:37:16,307 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/feed_forward2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,320 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/bypass_mid/Add_output_0" not specified 2023-11-23 15:37:16,326 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/self_attn2/Reshape_1_output_0" not specified 2023-11-23 15:37:16,336 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/Add_5_output_0" not specified 2023-11-23 15:37:16,347 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/conv_module2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,354 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/Add_6_output_0" not specified 2023-11-23 15:37:16,368 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/0/layers.1/feed_forward3/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,382 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/downsample/ReduceSum_output_0" not specified 2023-11-23 15:37:16,392 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/encoder_pos/Unsqueeze_11_output_0" not specified 2023-11-23 15:37:16,416 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/feed_forward1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,432 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/Add_output_0" not specified 2023-11-23 15:37:16,450 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/nonlin_attention/Mul_4_output_0" not specified 2023-11-23 15:37:16,458 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/Add_1_output_0" not specified 2023-11-23 15:37:16,465 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/self_attn1/Reshape_1_output_0" not specified 2023-11-23 15:37:16,470 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/Add_2_output_0" not specified 2023-11-23 15:37:16,485 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/conv_module1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,495 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/Add_3_output_0" not specified 2023-11-23 15:37:16,516 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/feed_forward2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,537 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/bypass_mid/Add_output_0" not specified 2023-11-23 15:37:16,544 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/self_attn2/Reshape_1_output_0" not specified 2023-11-23 15:37:16,550 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/Add_5_output_0" not specified 2023-11-23 15:37:16,564 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/conv_module2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,574 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/Add_6_output_0" not specified 2023-11-23 15:37:16,599 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/feed_forward3/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,623 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/0/bypass/Add_output_0" not specified 2023-11-23 15:37:16,655 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/feed_forward1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,672 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/Add_output_0" not specified 2023-11-23 15:37:16,689 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/nonlin_attention/Mul_4_output_0" not specified 2023-11-23 15:37:16,698 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/Add_1_output_0" not specified 2023-11-23 15:37:16,705 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/self_attn1/Reshape_1_output_0" not specified 2023-11-23 15:37:16,710 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/Add_2_output_0" not specified 2023-11-23 15:37:16,726 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/conv_module1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,736 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/Add_3_output_0" not specified 2023-11-23 15:37:16,756 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/feed_forward2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,777 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/bypass_mid/Add_output_0" not specified 2023-11-23 15:37:16,784 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/self_attn2/Reshape_1_output_0" not specified 2023-11-23 15:37:16,790 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/Add_5_output_0" not specified 2023-11-23 15:37:16,805 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/conv_module2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,901 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/Add_6_output_0" not specified 2023-11-23 15:37:16,926 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/1/encoder/1/feed_forward3/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:16,951 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/downsample/ReduceSum_output_0" not specified 2023-11-23 15:37:16,961 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/encoder_pos/Unsqueeze_11_output_0" not specified 2023-11-23 15:37:16,984 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/feed_forward1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,001 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/Add_output_0" not specified 2023-11-23 15:37:17,019 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/nonlin_attention/Mul_4_output_0" not specified 2023-11-23 15:37:17,028 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/Add_1_output_0" not specified 2023-11-23 15:37:17,035 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/self_attn1/Reshape_1_output_0" not specified 2023-11-23 15:37:17,041 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/Add_2_output_0" not specified 2023-11-23 15:37:17,057 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/conv_module1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,067 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/Add_3_output_0" not specified 2023-11-23 15:37:17,089 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/feed_forward2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,110 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/bypass_mid/Add_output_0" not specified 2023-11-23 15:37:17,118 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/self_attn2/Reshape_1_output_0" not specified 2023-11-23 15:37:17,124 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/Add_5_output_0" not specified 2023-11-23 15:37:17,139 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/conv_module2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,149 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/Add_6_output_0" not specified 2023-11-23 15:37:17,176 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/feed_forward3/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,201 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/0/bypass/Add_output_0" not specified 2023-11-23 15:37:17,234 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/feed_forward1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,251 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/Add_output_0" not specified 2023-11-23 15:37:17,270 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/nonlin_attention/Mul_4_output_0" not specified 2023-11-23 15:37:17,279 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/Add_1_output_0" not specified 2023-11-23 15:37:17,286 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/self_attn1/Reshape_1_output_0" not specified 2023-11-23 15:37:17,292 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/Add_2_output_0" not specified 2023-11-23 15:37:17,308 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/conv_module1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,319 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/Add_3_output_0" not specified 2023-11-23 15:37:17,340 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/feed_forward2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,361 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/bypass_mid/Add_output_0" not specified 2023-11-23 15:37:17,369 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/self_attn2/Reshape_1_output_0" not specified 2023-11-23 15:37:17,374 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/Add_5_output_0" not specified 2023-11-23 15:37:17,390 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/conv_module2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,400 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/Add_6_output_0" not specified 2023-11-23 15:37:17,426 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/2/encoder/1/feed_forward3/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,451 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/downsample/ReduceSum_output_0" not specified 2023-11-23 15:37:17,468 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/encoder_pos/Unsqueeze_11_output_0" not specified 2023-11-23 15:37:17,492 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/feed_forward1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,509 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/Add_output_0" not specified 2023-11-23 15:37:17,527 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/nonlin_attention/Mul_4_output_0" not specified 2023-11-23 15:37:17,536 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/Add_1_output_0" not specified 2023-11-23 15:37:17,545 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/self_attn1/Reshape_1_output_0" not specified 2023-11-23 15:37:17,552 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/Add_2_output_0" not specified 2023-11-23 15:37:17,568 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/conv_module1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,578 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/Add_3_output_0" not specified 2023-11-23 15:37:17,599 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/feed_forward2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,621 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/bypass_mid/Add_output_0" not specified 2023-11-23 15:37:17,629 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/self_attn2/Reshape_1_output_0" not specified 2023-11-23 15:37:17,636 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/Add_5_output_0" not specified 2023-11-23 15:37:17,652 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/conv_module2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,663 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/Add_6_output_0" not specified 2023-11-23 15:37:17,688 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/feed_forward3/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,713 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/0/bypass/Add_output_0" not specified 2023-11-23 15:37:17,753 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/feed_forward1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,770 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/Add_output_0" not specified 2023-11-23 15:37:17,788 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/nonlin_attention/Mul_4_output_0" not specified 2023-11-23 15:37:17,798 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/Add_1_output_0" not specified 2023-11-23 15:37:17,807 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/self_attn1/Reshape_1_output_0" not specified 2023-11-23 15:37:17,814 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/Add_2_output_0" not specified 2023-11-23 15:37:17,831 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/conv_module1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,842 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/Add_3_output_0" not specified 2023-11-23 15:37:17,864 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/feed_forward2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,887 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/bypass_mid/Add_output_0" not specified 2023-11-23 15:37:17,896 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/self_attn2/Reshape_1_output_0" not specified 2023-11-23 15:37:17,904 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/Add_5_output_0" not specified 2023-11-23 15:37:17,923 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/conv_module2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,934 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/Add_6_output_0" not specified 2023-11-23 15:37:17,963 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/3/encoder/1/feed_forward3/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:17,989 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/downsample/ReduceSum_output_0" not specified 2023-11-23 15:37:18,001 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/encoder_pos/Unsqueeze_11_output_0" not specified 2023-11-23 15:37:18,025 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/feed_forward1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,042 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/Add_output_0" not specified 2023-11-23 15:37:18,062 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/nonlin_attention/Mul_4_output_0" not specified 2023-11-23 15:37:18,072 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/Add_1_output_0" not specified 2023-11-23 15:37:18,080 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/self_attn1/Reshape_1_output_0" not specified 2023-11-23 15:37:18,086 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/Add_2_output_0" not specified 2023-11-23 15:37:18,102 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/conv_module1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,113 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/Add_3_output_0" not specified 2023-11-23 15:37:18,134 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/feed_forward2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,156 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/bypass_mid/Add_output_0" not specified 2023-11-23 15:37:18,164 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/self_attn2/Reshape_1_output_0" not specified 2023-11-23 15:37:18,171 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/Add_5_output_0" not specified 2023-11-23 15:37:18,187 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/conv_module2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,198 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/Add_6_output_0" not specified 2023-11-23 15:37:18,225 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/feed_forward3/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,251 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/0/bypass/Add_output_0" not specified 2023-11-23 15:37:18,285 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/feed_forward1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,302 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/Add_output_0" not specified 2023-11-23 15:37:18,323 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/nonlin_attention/Mul_4_output_0" not specified 2023-11-23 15:37:18,332 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/Add_1_output_0" not specified 2023-11-23 15:37:18,341 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/self_attn1/Reshape_1_output_0" not specified 2023-11-23 15:37:18,347 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/Add_2_output_0" not specified 2023-11-23 15:37:18,363 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/conv_module1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,374 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/Add_3_output_0" not specified 2023-11-23 15:37:18,398 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/feed_forward2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,419 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/bypass_mid/Add_output_0" not specified 2023-11-23 15:37:18,428 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/self_attn2/Reshape_1_output_0" not specified 2023-11-23 15:37:18,434 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/Add_5_output_0" not specified 2023-11-23 15:37:18,451 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/conv_module2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,462 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/Add_6_output_0" not specified 2023-11-23 15:37:18,488 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/4/encoder/1/feed_forward3/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,515 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/downsample/ReduceSum_output_0" not specified 2023-11-23 15:37:18,526 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/encoder_pos/Unsqueeze_11_output_0" not specified 2023-11-23 15:37:18,553 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/feed_forward1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,571 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/Add_output_0" not specified 2023-11-23 15:37:18,591 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/nonlin_attention/Mul_4_output_0" not specified 2023-11-23 15:37:18,601 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/Add_1_output_0" not specified 2023-11-23 15:37:18,610 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/self_attn1/Reshape_1_output_0" not specified 2023-11-23 15:37:18,616 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/Add_2_output_0" not specified 2023-11-23 15:37:18,633 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/conv_module1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,644 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/Add_3_output_0" not specified 2023-11-23 15:37:18,666 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/feed_forward2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,688 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/bypass_mid/Add_output_0" not specified 2023-11-23 15:37:18,697 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/self_attn2/Reshape_1_output_0" not specified 2023-11-23 15:37:18,704 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/Add_5_output_0" not specified 2023-11-23 15:37:18,721 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/conv_module2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,733 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/Add_6_output_0" not specified 2023-11-23 15:37:18,759 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/feed_forward3/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,786 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/0/bypass/Add_output_0" not specified 2023-11-23 15:37:18,821 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/feed_forward1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,838 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/Add_output_0" not specified 2023-11-23 15:37:18,858 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/nonlin_attention/Mul_4_output_0" not specified 2023-11-23 15:37:18,869 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/Add_1_output_0" not specified 2023-11-23 15:37:18,877 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/self_attn1/Reshape_1_output_0" not specified 2023-11-23 15:37:18,885 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/Add_2_output_0" not specified 2023-11-23 15:37:18,902 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/conv_module1/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,913 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/Add_3_output_0" not specified 2023-11-23 15:37:18,936 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/feed_forward2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:18,958 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/bypass_mid/Add_output_0" not specified 2023-11-23 15:37:18,967 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/self_attn2/Reshape_1_output_0" not specified 2023-11-23 15:37:18,974 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/Add_5_output_0" not specified 2023-11-23 15:37:18,992 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/conv_module2/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:19,003 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/Add_6_output_0" not specified 2023-11-23 15:37:19,030 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/encoder/5/encoder/1/feed_forward3/out_proj/Sub_2_output_0" not specified 2023-11-23 15:37:19,057 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/Transpose_1_output_0" not specified 2023-11-23 15:37:19,597 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/Squeeze_output_0" not specified 2023-11-23 15:37:19,663 INFO [onnx_quantizer.py:538] Quantization parameters for tensor:"/Tanh_output_0" not specified Ignore MatMul due to non constant B: /[/encoder/0/layers.0/self_attn_weights/MatMul_1] Ignore MatMul due to non constant B: /[/encoder/0/layers.0/self_attn_weights/MatMul] Ignore MatMul due to non constant B: /[/encoder/0/layers.0/nonlin_attention/MatMul] Ignore MatMul due to non constant B: /[/encoder/0/layers.0/self_attn1/MatMul] Ignore MatMul due to non constant B: /[/encoder/0/layers.0/self_attn2/MatMul] Ignore MatMul due to non constant B: /[/encoder/0/layers.1/self_attn_weights/MatMul_1] Ignore MatMul due to non constant B: /[/encoder/0/layers.1/self_attn_weights/MatMul] Ignore MatMul due to non constant B: /[/encoder/0/layers.1/nonlin_attention/MatMul] Ignore MatMul due to non constant B: /[/encoder/0/layers.1/self_attn1/MatMul] Ignore MatMul due to non constant B: /[/encoder/0/layers.1/self_attn2/MatMul] Ignore MatMul due to non constant B: /[/encoder/1/encoder/0/self_attn_weights/MatMul_1] Ignore MatMul due to non constant B: /[/encoder/1/encoder/0/self_attn_weights/MatMul] Ignore MatMul due to non constant B: /[/encoder/1/encoder/0/nonlin_attention/MatMul] Ignore MatMul due to non constant B: /[/encoder/1/encoder/0/self_attn1/MatMul] Ignore MatMul due to non constant B: /[/encoder/1/encoder/0/self_attn2/MatMul] Ignore MatMul due to non constant B: /[/encoder/1/encoder/1/self_attn_weights/MatMul_1] Ignore MatMul due to non constant B: /[/encoder/1/encoder/1/self_attn_weights/MatMul] Ignore MatMul due to non constant B: /[/encoder/1/encoder/1/nonlin_attention/MatMul] Ignore MatMul due to non constant B: /[/encoder/1/encoder/1/self_attn1/MatMul] Ignore MatMul due to non constant B: /[/encoder/1/encoder/1/self_attn2/MatMul] Ignore MatMul due to non constant B: /[/encoder/2/encoder/0/self_attn_weights/MatMul_1] Ignore MatMul due to non constant B: /[/encoder/2/encoder/0/self_attn_weights/MatMul] Ignore MatMul due to non constant B: /[/encoder/2/encoder/0/nonlin_attention/MatMul] Ignore MatMul due to non constant B: /[/encoder/2/encoder/0/self_attn1/MatMul] Ignore MatMul due to non constant B: /[/encoder/2/encoder/0/self_attn2/MatMul] Ignore MatMul due to non constant B: /[/encoder/2/encoder/1/self_attn_weights/MatMul_1] Ignore MatMul due to non constant B: /[/encoder/2/encoder/1/self_attn_weights/MatMul] Ignore MatMul due to non constant B: /[/encoder/2/encoder/1/nonlin_attention/MatMul] Ignore MatMul due to non constant B: /[/encoder/2/encoder/1/self_attn1/MatMul] Ignore MatMul due to non constant B: /[/encoder/2/encoder/1/self_attn2/MatMul] Ignore MatMul due to non constant B: /[/encoder/3/encoder/0/self_attn_weights/MatMul_1] Ignore MatMul due to non constant B: /[/encoder/3/encoder/0/self_attn_weights/MatMul] Ignore MatMul due to non constant B: /[/encoder/3/encoder/0/nonlin_attention/MatMul] Ignore MatMul due to non constant B: /[/encoder/3/encoder/0/self_attn1/MatMul] Ignore MatMul due to non constant B: /[/encoder/3/encoder/0/self_attn2/MatMul] Ignore MatMul due to non constant B: /[/encoder/3/encoder/1/self_attn_weights/MatMul_1] Ignore MatMul due to non constant B: /[/encoder/3/encoder/1/self_attn_weights/MatMul] Ignore MatMul due to non constant B: /[/encoder/3/encoder/1/nonlin_attention/MatMul] Ignore MatMul due to non constant B: /[/encoder/3/encoder/1/self_attn1/MatMul] Ignore MatMul due to non constant B: /[/encoder/3/encoder/1/self_attn2/MatMul] Ignore MatMul due to non constant B: /[/encoder/4/encoder/0/self_attn_weights/MatMul_1] Ignore MatMul due to non constant B: /[/encoder/4/encoder/0/self_attn_weights/MatMul] Ignore MatMul due to non constant B: /[/encoder/4/encoder/0/nonlin_attention/MatMul] Ignore MatMul due to non constant B: /[/encoder/4/encoder/0/self_attn1/MatMul] Ignore MatMul due to non constant B: /[/encoder/4/encoder/0/self_attn2/MatMul] Ignore MatMul due to non constant B: /[/encoder/4/encoder/1/self_attn_weights/MatMul_1] Ignore MatMul due to non constant B: /[/encoder/4/encoder/1/self_attn_weights/MatMul] Ignore MatMul due to non constant B: /[/encoder/4/encoder/1/nonlin_attention/MatMul] Ignore MatMul due to non constant B: /[/encoder/4/encoder/1/self_attn1/MatMul] Ignore MatMul due to non constant B: /[/encoder/4/encoder/1/self_attn2/MatMul] Ignore MatMul due to non constant B: /[/encoder/5/encoder/0/self_attn_weights/MatMul_1] Ignore MatMul due to non constant B: /[/encoder/5/encoder/0/self_attn_weights/MatMul] Ignore MatMul due to non constant B: /[/encoder/5/encoder/0/nonlin_attention/MatMul] Ignore MatMul due to non constant B: /[/encoder/5/encoder/0/self_attn1/MatMul] Ignore MatMul due to non constant B: /[/encoder/5/encoder/0/self_attn2/MatMul] Ignore MatMul due to non constant B: /[/encoder/5/encoder/1/self_attn_weights/MatMul_1] Ignore MatMul due to non constant B: /[/encoder/5/encoder/1/self_attn_weights/MatMul] Ignore MatMul due to non constant B: /[/encoder/5/encoder/1/nonlin_attention/MatMul] Ignore MatMul due to non constant B: /[/encoder/5/encoder/1/self_attn1/MatMul] Ignore MatMul due to non constant B: /[/encoder/5/encoder/1/self_attn2/MatMul]
Logs of decode.py
2023-11-23 15:20:15,279 INFO [decode.py:791] Decoding started 2023-11-23 15:20:15,280 INFO [decode.py:797] Device: cpu 2023-11-23 15:20:15,283 INFO [decode.py:807] {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 50, 'reset_interval': 200, 'valid_interval': 3000, 'feature_dim': 80, 'subsampling_factor': 4, 'warm_step': 2000, 'env_info': {'k2-version': '1.24.3', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': '7ff6d891905ff482364f2d0015867b00d89dd8c7', 'k2-git-date': 'Fri Jun 16 12:10:37 2023', 'lhotse-version': '1.16.0.dev+git.aa073f6a.clean', 'torch-version': '1.13.1+cu117', 'torch-cuda-available': False, 'torch-cuda-version': '11.7', 'python-version': '3.9', 'icefall-git-branch': 'master', 'icefall-git-sha1': 'fc2df07-dirty', 'icefall-git-date': 'Wed Aug 16 20:02:41 2023', 'icefall-path': '/NAS1/sanjukta_repo_falcon1/zip_exp_6/icefall', 'k2-path': '/home/sanjukta/anaconda3/envs/zipf1/lib/python3.9/site-packages/k2/init.py', 'lhotse-path': '/NAS1/sanjukta_repo_falcon1/zip_exp_6/lhotse/lhotse/init.py', 'hostname': 'asus-System-Product-Name', 'IP address': '127.0.1.1'}, 'epoch': 41, 'iter': 0, 'avg': 1, 'use_averaged_model': False, 'exp_dir': PosixPath('zipformer/exp-ignite_via/25_10_2023'), 'bpe_model': 'data/ignite_via/20_10_2023/lang_bpe_500/bpe.model', 'lang_dir': PosixPath('data/lang_bpe_500'), 'decoding_method': 'greedy_search', 'beam_size': 4, 'beam': 20.0, 'ngram_lm_scale': 0.01, 'max_contexts': 8, 'max_states': 64, 'context_size': 2, 'max_sym_per_frame': 1, 'num_paths': 200, 'nbest_scale': 0.5, 'use_shallow_fusion': False, 'lm_type': 'rnn', 'lm_scale': 0.3, 'tokens_ngram': 2, 'backoff_id': 500, 'num_encoder_layers': '2,2,2,2,2,2', 'downsampling_factor': '1,2,4,8,4,2', 'feedforward_dim': '512,768,768,768,768,768', 'num_heads': '4,4,4,8,4,4', 'encoder_dim': '192,256,256,256,256,256', 'query_head_dim': '32', 'value_head_dim': '12', 'pos_head_dim': '4', 'pos_dim': 48, 'encoder_unmasked_dim': '192,192,192,192,192,192', 'cnn_module_kernel': '31,31,15,15,15,31', 'decoder_dim': 512, 'joiner_dim': 512, 'causal': False, 'chunk_size': '16', 'left_context_frames': '64', 'use_transducer': True, 'use_ctc': False, 'full_libri': True, 'mini_libri': False, 'manifest_dir': PosixPath('data/test/ignite_via/via/22_11_2023_2.0/fbank'), 'max_duration': 200.0, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'drop_last': True, 'return_cuts': True, 'num_workers': 2, 'enable_spec_aug': True, 'spec_aug_time_warp_factor': 80, 'enable_musan': True, 'input_strategy': 'PrecomputedFeatures', 'lm_vocab_size': 500, 'lm_epoch': 7, 'lm_avg': 1, 'lm_exp_dir': None, 'rnn_lm_embedding_dim': 2048, 'rnn_lm_hidden_dim': 2048, 'rnn_lm_num_layers': 3, 'rnn_lm_tie_weights': True, 'transformer_lm_exp_dir': None, 'transformer_lm_dim_feedforward': 2048, 'transformer_lm_encoder_dim': 768, 'transformer_lm_embedding_dim': 768, 'transformer_lm_nhead': 8, 'transformer_lm_num_layers': 16, 'transformer_lm_tie_weights': True, 'res_dir': PosixPath('zipformer/exp-ignite_via/25_10_2023/greedy_search'), 'suffix': 'epoch-40-avg-1-context-2-max-sym-per-frame-1', 'blank_id': 0, 'unk_id': 2, 'vocab_size': 500} 2023-11-23 15:20:15,283 INFO [decode.py:809] About to create model 2023-11-23 15:20:15,521 INFO [checkpoint.py:112] Loading checkpoint from zipformer/exp-ignite_via/25_10_2023/epoch-40.pt 2023-11-23 15:20:15,624 INFO [decode.py:958] Number of model parameters: 23285615 2023-11-23 15:20:15,624 INFO [asr_datamodule.py:465] About to get test-clean cuts 2023-11-23 15:20:53,647 INFO [decode.py:675] batch 0/?, cuts processed until now is 1 2023-11-23 15:21:20,039 INFO [zipformer.py:1728] name=None, attn_weights_entropy = tensor([5.3356, 4.6086, 4.9479, 5.5353]) 2023-11-23 15:21:33,954 INFO [decode.py:691] The transcripts are stored in zipformer/exp-ignite_via/25_10_2023/greedy_search/recogs-test-clean-greedy_search-epoch-40-avg-1-context-2-max-sym-per-frame-1.txt 2023-11-23 15:21:33,958 INFO [utils.py:564] [test-clean-greedy_search] %WER 1445.64% [2154 / 149, 2005 ins, 0 del, 149 sub ] 2023-11-23 15:21:33,973 INFO [decode.py:704] Wrote detailed error stats to zipformer/exp-ignite_via/25_10_2023/greedy_search/errs-test-clean-greedy_search-epoch-40-avg-1-context-2-max-sym-per-frame-1.txt 2023-11-23 15:21:33,974 INFO [decode.py:720] For test-clean, WER of different settings are: greedy_search 1445.64 best for test-clean
2023-11-23 15:21:33,974 INFO [decode.py:1000] Done!
Did you get any chance to look into the logs ?
The logs look correct.
Does pretrained.py give correct result?
By default decode.py uses pre-computed features while onnx_pretrained.py and pretrained.py compute features on the fly.
Output of pretrained.py:
θBΩə θF μFOCSQə βMλə OBλə C∞ψθLλJ∞ζə ζəλə θBΩə BC ΩK@ə γC μFOCSQə C∞βJO!ə ΩK@ə CSOφBθəλ@ə βθF @Jρə γKOə L∞əμC K∞@ə BC βλMεə θLλə J∞@ə !F OMμə εC γKOə γə!ə ψC @C@ə∞!ə !əλ∞ə Mβə ζMλC Kεə BC M@Cγəμə ζəλə ΩKμL θBΩə OK∞ə θF ΩCθəλə εC
The output is almost similar to decode.py
The problem of decoded result getting blanked out is not happening here.
I would suggest that you use the same feature input for the torch model and the onnx model and compare the output of the encoder, decoder, and joiner and check which part is incorrect.
This is the first time we have seen a mismatch between the torch model and the onnx model.