yolov5-seg-ncnn icon indicating copy to clipboard operation
yolov5-seg-ncnn copied to clipboard

Published 20 hours ago • verified publisher icon FeiGeChuanShu

pt-->ncnn

Open noinoivip911 opened this issue 2 years ago • 9 comments

Hi, could you tell me the way to convert pretrained weight from pt to ncnn? Thank you.

noinoivip911 avatar Sep 21 '22 03:09 noinoivip911

Hi, could you tell me the way to convert pretrained weight from pt to ncnn? Thank you.

    1. export the torchscript file by export.py.
    1. use pnnx tool to convert the torchscript to ncnn model.

FeiGeChuanShu avatar Sep 21 '22 08:09 FeiGeChuanShu

// When I converted torchscript --> ncnn by pnnx tool, this error had occurred. Could you show me the reason of this error? I've used pre-trained YOLOv5n_seg weights. My command line to convert:

./pnnx yolov5n-seg.torchscript inputshape=[1,3,640,640] inputshape2=[1,3,320,320]

Thanks

My error:

pnnxparam = /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/yolov5n-seg.pnnx.param pnnxbin = /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/yolov5n-seg.pnnx.bin pnnxpy = /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/yolov5n-seg_pnnx.py ncnnparam = /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/yolov5n-seg.ncnn.param ncnnbin = /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/yolov5n-seg.ncnn.bin ncnnpy = /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/yolov5n-seg_ncnn.py optlevel = 2 device = cpu inputshape = [1,3,640,640]f32 inputshape2 = [1,3,320,320]f32 customop = moduleop = ############# pass_level0 inline module = models.common.Bottleneck inline module = models.common.C3 inline module = models.common.Concat inline module = models.common.Conv inline module = models.common.Proto inline module = models.common.SPPF inline module = models.yolo.Segment inline module = models.common.Bottleneck inline module = models.common.C3 inline module = models.common.Concat inline module = models.common.Conv inline module = models.common.Proto inline module = models.common.SPPF inline module = models.yolo.Segment 187 188 190 input.8 198 input.6 205 input.20 208 209 210 212 213 input.12 217 218 220 input.10 228 input.16 236 input.18 239 240 241 246 input.14 249 250 251 253 254 input.24 258 259 261 input.22 269 input.28 278 input.74 281 282 283 288 input.4 291 292 293 298 input.26 301 302 303 305 306 input.32 310 311 313 input.30 321 input.42 328 input.34 331 332 333 335 336 input.36 340 341 347 input.38 349 350 351 input.46 355 356 358 input.40 132 input.44 369 input.52 375 input.48 378 379 381 382 input.56 386 387 389 input.50 143 input.54 400 input.60 406 input.58 409 410 412 413 input.64 417 418 420 421 input.62 431 input.68 437 input.66 440 441 443 444 input.72 448 449 451 452 input.70 462 input.5 468 input.76 471 472 474 475 input.3 479 480 481 482 483 484 485 486 487 488 489 510 input.2 512 514 input.1 517 518 519 bs.1 ny.1 nx.1 531 533 534 xy.1 wh.1 conf.1 mask.1 541 542 543 544 xy0.1 546 650 547 548 wh0.1 550 y.1 553 554 557 558 bs0.1 ny0.1 nx0.1 570 572 573 xy1.1 wh1.1 conf0.1 mask0.1 580 670 581 582 672 583 xy2.1 585 675 586 587 wh2.1 589 y0.1 678 592 593 596 597 bs1.1 ny1.1 nx1.1 609 611 612 xy3.1 wh3.1 conf1.1 mask1.1 619 696 620 621 698 622 xy4.1 624 701 625 626 wh4.1 628 y1.1 704 631 632 635 637 172 173
terminate` called after throwing an instance of 'std::runtime_error' what(): The following operation failed in the TorchScript interpreter. Traceback of TorchScript, serialized code (most recent call last): File "code/torch/models/yolo.py", line 125, in fallback_function xy1, wh1, conf0, mask0, = _34 _35 = torch.mul(torch.sigmoid(xy1), CONSTANTS.c0) xy2 = torch.mul(torch.add(_35, CONSTANTS.c5), torch.select(CONSTANTS.c2, 0, 1)) ~~~~~~~~~ <--- HERE _36 = torch.mul(torch.sigmoid(wh1), CONSTANTS.c0) wh2 = torch.mul(torch.pow(_36, 2), CONSTANTS.c6) Traceback of TorchScript, original code (most recent call last): /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/models/yolo.py(69): forward /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/models/yolo.py(105): forward /home/vtcc-thuynt/miniconda3/lib/python3.9/site-packages/torch/nn/modules/module.py(1098): _slow_forward /home/vtcc-thuynt/miniconda3/lib/python3.9/site-packages/torch/nn/modules/module.py(1110): _call_impl /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/models/yolo.py(121): _forward_once /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/models/yolo.py(209): forward /home/vtcc-thuynt/miniconda3/lib/python3.9/site-packages/torch/nn/modules/module.py(1098): _slow_forward /home/vtcc-thuynt/miniconda3/lib/python3.9/site-packages/torch/nn/modules/module.py(1110): _call_impl /home/vtcc-thuynt/miniconda3/lib/python3.9/site-packages/torch/jit/_trace.py(958): trace_module /home/vtcc-thuynt/miniconda3/lib/python3.9/site-packages/torch/jit/_trace.py(741): trace /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/export.py(120): export_torchscript /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/export.py(104): outer_func /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/export.py(528): run /home/vtcc-thuynt/miniconda3/lib/python3.9/site-packages/torch/autograd/grad_mode.py(27): decorate_context /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/export.py(612): main /home/vtcc-thuynt/Desktop/yolov5_segment/yolov5/export.py(617): RuntimeError: The size of tensor a (20) must match the size of tensor b (40) at non-singleton dimension 3

Aborted (core dumped)

noinoivip911 avatar Sep 22 '22 02:09 noinoivip911

@noinoivip911 Have you solved your problem? I have the same issue. I'm using this command for exporting:

pnnx yolov5s-seg.torchscript inputshape=[1,3,640,640] inputshape2=[1,3,320,320]

sctrueew avatar Oct 01 '22 17:10 sctrueew

@noinoivip911 Hi, Did you convert to ncnn model successfully?

sctrueew avatar Oct 05 '22 15:10 sctrueew

have you solved it? I have the same issue, Tks

nqthai309 avatar Oct 17 '22 09:10 nqthai309

Hi, could you tell me the way to convert pretrained weight from pt to ncnn? Thank you.

    1. export the torchscript file by export.py.
    1. use pnnx tool to convert the torchscript to ncnn model.

0. environment

Ubuntu 18.04.6 LTS
torch                     1.8.0+cpu
torchaudio                0.8.0
torchvision               0.9.0+cpu
libtorch                  libtorch-cxx11-abi-shared-with-deps-1.8.0+cpu.zip
pnnx                      ncnn-master/tools/pnnx/     (ncnn-master 2023.1.16 18:40)
gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
cmake version 3.18.0

1. pt2torchscript

I trained an image segment model tpadseg_v0.0.1.pt based on yolov5m-seg.pt by yolov5-7.0，and convert to torchscript format through the following command :

$ python export.py --weights /home/tianzx/ai_model/seg/tpadseg_v0.0.1.pt --include torchscript

2. torchscript -> pnnx ncnn

2.1 move torchscript

$ cd /home/tianzx/ai_model/seg/
$ mkdir pnnx
$ mv tpadseg_v0.0.1.torchscript pnnx/

2.2 convert torchscript to pnnx ncnn

  • confirm pnnx
$ cd /home/tianzx/app/pnnx/ncnn-master/tools/pnnx/build/install/bin/
$ ./pnnx
Usage: pnnx [model.pt] [(key=value)...]
  pnnxparam=model.pnnx.param
  pnnxbin=model.pnnx.bin
  pnnxpy=model_pnnx.py
  pnnxonnx=model.pnnx.onnx
  ncnnparam=model.ncnn.param
  ncnnbin=model.ncnn.bin
  ncnnpy=model_ncnn.py
  fp16=1
  optlevel=2
  device=cpu/gpu
  inputshape=[1,3,224,224],...
  inputshape2=[1,3,320,320],...
  customop=/home/nihui/.cache/torch_extensions/fused/fused.so,...
  moduleop=models.common.Focus,models.yolo.Detect,...
Sample usage: pnnx mobilenet_v2.pt inputshape=[1,3,224,224]
              pnnx yolov5s.pt inputshape=[1,3,640,640]f32 inputshape2=[1,3,320,320]f32 device=gpu moduleop=models.common.Focus,models.yolo.Detect
  • convert torchscript
$ ./pnnx /home/tianzx/ai_model/seg/pnnx/tpadseg_v0.0.1.torchscript inputshape=[1,3,640,640]
pnnxparam = /home/tianzx/ai_model/seg/pnnx/tpadseg_v0.0.1.pnnx.param
pnnxbin = /home/tianzx/ai_model/seg/pnnx/tpadseg_v0.0.1.pnnx.bin
pnnxpy = /home/tianzx/ai_model/seg/pnnx/tpadseg_v0.0.1_pnnx.py
pnnxonnx = /home/tianzx/ai_model/seg/pnnx/tpadseg_v0.0.1.pnnx.onnx
ncnnparam = /home/tianzx/ai_model/seg/pnnx/tpadseg_v0.0.1.ncnn.param
ncnnbin = /home/tianzx/ai_model/seg/pnnx/tpadseg_v0.0.1.ncnn.bin
ncnnpy = /home/tianzx/ai_model/seg/pnnx/tpadseg_v0.0.1_ncnn.py
fp16 = 1
optlevel = 2
device = cpu
inputshape = [1,3,640,640]f32
inputshape2 = 
customop = 
moduleop = 
############# pass_level0
inline module = models.common.Bottleneck
inline module = models.common.C3
inline module = models.common.Concat
inline module = models.common.Conv
inline module = models.common.Proto
inline module = models.common.SPPF
inline module = models.yolo.Segment
inline module = models.common.Bottleneck
inline module = models.common.C3
inline module = models.common.Concat
inline module = models.common.Conv
inline module = models.common.Proto
inline module = models.common.SPPF
inline module = models.yolo.Segment

----------------

############# pass_level1
no attribute value
no attribute value
unknown Parameter value kind prim::Constant of TensorType, t.dim = 5
unknown Parameter value kind prim::Constant of TensorType, t.dim = 5
unknown Parameter value kind prim::Constant of TensorType, t.dim = 5
unknown Parameter value kind prim::Constant of TensorType, t.dim = 5
unknown Parameter value kind prim::Constant of TensorType, t.dim = 5
unknown Parameter value kind prim::Constant of TensorType, t.dim = 1
unknown Parameter value kind prim::Constant of TensorType, t.dim = 5
no attribute value
unknown Parameter value kind prim::Constant of TensorType, t.dim = 1
unknown Parameter value kind prim::Constant of TensorType, t.dim = 1
############# pass_level2
############# pass_level3
assign unique operator name pnnx_unique_0 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_1 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_2 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_3 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_4 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_5 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_6 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_7 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_8 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_9 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_10 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_11 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_12 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_13 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_14 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_15 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_16 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_17 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_18 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_19 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_20 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_21 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_22 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_23 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_24 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_25 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_26 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_27 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_28 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_29 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_30 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_31 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_32 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_33 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_34 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_35 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_36 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_37 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_38 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_39 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_40 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_41 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_42 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_43 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_44 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_45 to model.9.m
assign unique operator name pnnx_unique_46 to model.9.m
assign unique operator name pnnx_unique_47 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_48 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_49 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_50 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_51 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_52 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_53 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_54 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_55 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_56 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_57 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_58 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_59 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_60 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_61 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_62 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_63 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_64 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_65 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_66 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_67 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_68 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_69 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_70 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_71 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_72 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_73 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_74 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_75 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_76 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_77 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_78 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_79 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_80 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_81 to model.24.proto.cv3.act
assign unique operator name pnnx_unique_82 to model.24.proto.cv3.act
############# pass_level4
############# pass_level5
############# pass_ncnn
select along batch axis 0 is not supported
select along batch axis 0 is not supported
select along batch axis 0 is not supported
ignore Crop select_0 param dim=0
ignore Crop select_0 param index=0
ignore Crop select_1 param dim=0
ignore Crop select_1 param index=1
ignore Crop select_2 param dim=0
ignore Crop select_2 param index=2

The convert from torchscript to pnnx ncnn do not work correct. Could you help me out ? @FeiGeChuanShu

Thanks in advance.

By the way, the files can be find in pnnx.zip from Baidu Disk or Google Drive.

$ tree .
.
├── tpadseg_v0.0.1.pt
└── tpadseg_v0.0.1.torchscript

0 directories, 2 files

Baidu Disk Download link : https://pan.baidu.com/s/17lxgBoNa3Y7t_qg-rMDENw?pwd=5pff Extraction code: 5pff

Google Drive: https://drive.google.com/drive/folders/1NZKFMDwOP113NKLtoPPyzRL9F9Tl7dAy?usp=sharing

Digital2Slave avatar Jan 17 '23 03:01 Digital2Slave

./pnnx yolov5n-seg.torchscript inputshape=[1,3,640,640]

@noinoivip911

You should use ./pnnx yolov5n-seg.torchscript inputshape=[1,3,640,640]

Digital2Slave avatar Jan 18 '23 05:01 Digital2Slave

@noinoivip911 Have you solved your problem? I have the same issue. I'm using this command for exporting:

pnnx yolov5s-seg.torchscript inputshape=[1,3,640,640] inputshape2=[1,3,320,320]

@sctrueew Check ./pnnx yolov5s-seg.torchscript inputshape=[1,3,640,640]

Digital2Slave avatar Jan 18 '23 05:01 Digital2Slave

have you solved it? I have the same issue, Tks

@nqthai309 Do you find solution :question:

Digital2Slave avatar Jan 18 '23 05:01 Digital2Slave