[WIP]fix trt for swin inference

[littletomatodonkey]

For faster inference speed in swin-series model, you can

1. generate shape info file

python3.7 python/predict_cls.py \
    -c configs/inference_cls.yaml \
    -o Global.batch_size=1 \
    -o Global.use_tensorrt=True \
    -o Global.collect_shape=True \
    -o Global.shape_range_info_filename="ppcls_infer_shape_info_bs1.pbtxt"

2. inference

python3.7 python/predict_cls.py \
    -c configs/inference_cls.yaml \
    -o Global.batch_size=1 \
    -o Global.use_tensorrt=True \
    -o Global.collect_shape=False \
    -o Global.shape_range_info_filename="ppcls_infer_shape_info_bs1.pbtxt"

3. Some performance data on V100 GPU + trt7.2.3.4 + cudnn8.1.1

• model: swin_tiny
• batch_size: 1

wo trt (just GPU)
[2022/05/24 11:14:52] root INFO:  preprocess_time(ms): 3.4285, inference_time(ms): 7.6241, postprocess_time(ms): 0.1837
trt + fp32 load file, sub graph size=3
[2022/05/24 10:50:52] root INFO:  preprocess_time(ms): 3.5497, inference_time(ms): 5.1654, postprocess_time(ms): 0.1823

trt + fp32 load file, sub graph size=30
[2022/05/24 11:01:48] root INFO:  preprocess_time(ms): 3.4677, inference_time(ms): 5.2271, postprocess_time(ms): 0.1823

trt + fp32 not load file, sub graph size=15
[2022/05/24 11:00:17] root INFO:  preprocess_time(ms): 3.3461, inference_time(ms): 8.5095, postprocess_time(ms): 0.174

trt + fp32 not load file, sub graph size=30
[2022/05/24 11:00:54] root INFO:  preprocess_time(ms): 3.6853, inference_time(ms): 10.4525, postprocess_time(ms): 0.2137

[paddle-bot-old[bot]]

Thanks for your contribution!

[paddle-bot[bot]]

Automatically closed by Paddle-bot.