nguyenvanhoang7398

I'm encountering this error even on a machine with 4 16GB GPUs. When this happened I checked the GPU consumption and it was very low, so it couldn't be that...

Thank you, I have changed the --opt-length flag in the `trtexec` command as below ``` trtexec --onnx=m2m100_418M-decoder.onnx --verbose --minShapes=input_ids:5x1,encoder_attention_mask:5x1,pkv_0:5x16x1x64,pkv_1:5x16x1x64,pkv_2:5x16x1x64,pkv_3:5x16x1x64,pkv_4:5x16x1x64,pkv_5:5x16x1x64,pkv_6:5x16x1x64,pkv_7:5x16x1x64,pkv_8:5x16x1x64,pkv_9:5x16x1x64,pkv_10:5x16x1x64,pkv_11:5x16x1x64,pkv_12:5x16x1x64,pkv_13:5x16x1x64,pkv_14:5x16x1x64,pkv_15:5x16x1x64,pkv_16:5x16x1x64,pkv_17:5x16x1x64,pkv_18:5x16x1x64,pkv_19:5x16x1x64,pkv_20:5x16x1x64,pkv_21:5x16x1x64,pkv_22:5x16x1x64,pkv_23:5x16x1x64,pkv_24:5x16x1x64,pkv_25:5x16x1x64,pkv_26:5x16x1x64,pkv_27:5x16x1x64,pkv_28:5x16x1x64,pkv_29:5x16x1x64,pkv_30:5x16x1x64,pkv_31:5x16x1x64,pkv_32:5x16x1x64,pkv_33:5x16x1x64,pkv_34:5x16x1x64,pkv_35:5x16x1x64,pkv_36:5x16x1x64,pkv_37:5x16x1x64,pkv_38:5x16x1x64,pkv_39:5x16x1x64,pkv_40:5x16x1x64,pkv_41:5x16x1x64,pkv_42:5x16x1x64,pkv_43:5x16x1x64,pkv_44:5x16x1x64,pkv_45:5x16x1x64,pkv_46:5x16x1x64,pkv_47:5x16x1x64 --optShapes=input_ids:5x100,encoder_attention_mask:5x100,pkv_0:5x16x100x64,pkv_1:5x16x100x64,pkv_2:5x16x100x64,pkv_3:5x16x100x64,pkv_4:5x16x100x64,pkv_5:5x16x100x64,pkv_6:5x16x100x64,pkv_7:5x16x100x64,pkv_8:5x16x100x64,pkv_9:5x16x100x64,pkv_10:5x16x100x64,pkv_11:5x16x100x64,pkv_12:5x16x100x64,pkv_13:5x16x100x64,pkv_14:5x16x100x64,pkv_15:5x16x100x64,pkv_16:5x16x100x64,pkv_17:5x16x100x64,pkv_18:5x16x100x64,pkv_19:5x16x100x64,pkv_20:5x16x100x64,pkv_21:5x16x100x64,pkv_22:5x16x100x64,pkv_23:5x16x100x64,pkv_24:5x16x100x64,pkv_25:5x16x100x64,pkv_26:5x16x100x64,pkv_27:5x16x100x64,pkv_28:5x16x100x64,pkv_29:5x16x100x64,pkv_30:5x16x100x64,pkv_31:5x16x100x64,pkv_32:5x16x100x64,pkv_33:5x16x100x64,pkv_34:5x16x100x64,pkv_35:5x16x100x64,pkv_36:5x16x100x64,pkv_37:5x16x100x64,pkv_38:5x16x100x64,pkv_39:5x16x100x64,pkv_40:5x16x100x64,pkv_41:5x16x100x64,pkv_42:5x16x100x64,pkv_43:5x16x100x64,pkv_44:5x16x100x64,pkv_45:5x16x100x64,pkv_46:5x16x100x64,pkv_47:5x16x100x64 --maxShapes=input_ids:80x200,encoder_attention_mask:80x200,pkv_0:80x16x200x64,pkv_1:80x16x200x64,pkv_2:80x16x200x64,pkv_3:80x16x200x64,pkv_4:80x16x200x64,pkv_5:80x16x200x64,pkv_6:80x16x200x64,pkv_7:80x16x200x64,pkv_8:80x16x200x64,pkv_9:80x16x200x64,pkv_10:80x16x200x64,pkv_11:80x16x200x64,pkv_12:80x16x200x64,pkv_13:80x16x200x64,pkv_14:80x16x200x64,pkv_15:80x16x200x64,pkv_16:80x16x200x64,pkv_17:80x16x200x64,pkv_18:80x16x200x64,pkv_19:80x16x200x64,pkv_20:80x16x200x64,pkv_21:80x16x200x64,pkv_22:80x16x200x64,pkv_23:80x16x200x64,pkv_24:80x16x200x64,pkv_25:80x16x200x64,pkv_26:80x16x200x64,pkv_27:80x16x200x64,pkv_28:80x16x200x64,pkv_29:80x16x200x64,pkv_30:80x16x200x64,pkv_31:80x16x200x64,pkv_32:80x16x200x64,pkv_33:80x16x200x64,pkv_34:80x16x200x64,pkv_35:80x16x200x64,pkv_36:80x16x200x64,pkv_37:80x16x200x64,pkv_38:80x16x200x64,pkv_39:80x16x200x64,pkv_40:80x16x200x64,pkv_41:80x16x200x64,pkv_42:80x16x200x64,pkv_43:80x16x200x64,pkv_44:80x16x200x64,pkv_45:80x16x200x64,pkv_46:80x16x200x64,pkv_47:80x16x200x64 --shapes=input_ids:5x1,encoder_attention_mask:5x1,pkv_0:5x16x1x64,pkv_1:5x16x1x64,pkv_2:5x16x1x64,pkv_3:5x16x1x64,pkv_4:5x16x1x64,pkv_5:5x16x1x64,pkv_6:5x16x1x64,pkv_7:5x16x1x64,pkv_8:5x16x1x64,pkv_9:5x16x1x64,pkv_10:5x16x1x64,pkv_11:5x16x1x64,pkv_12:5x16x1x64,pkv_13:5x16x1x64,pkv_14:5x16x1x64,pkv_15:5x16x1x64,pkv_16:5x16x1x64,pkv_17:5x16x1x64,pkv_18:5x16x1x64,pkv_19:5x16x1x64,pkv_20:5x16x1x64,pkv_21:5x16x1x64,pkv_22:5x16x1x64,pkv_23:5x16x1x64,pkv_24:5x16x1x64,pkv_25:5x16x1x64,pkv_26:5x16x1x64,pkv_27:5x16x1x64,pkv_28:5x16x1x64,pkv_29:5x16x1x64,pkv_30:5x16x1x64,pkv_31:5x16x1x64,pkv_32:5x16x1x64,pkv_33:5x16x1x64,pkv_34:5x16x1x64,pkv_35:5x16x1x64,pkv_36:5x16x1x64,pkv_37:5x16x1x64,pkv_38:5x16x1x64,pkv_39:5x16x1x64,pkv_40:5x16x1x64,pkv_41:5x16x1x64,pkv_42:5x16x1x64,pkv_43:5x16x1x64,pkv_44:5x16x1x64,pkv_45:5x16x1x64,pkv_46:5x16x1x64,pkv_47:5x16x1x64 ``` Here are the params: ``` min_batch =...