PaddleSlim
PaddleSlim copied to clipboard
PaddlePaddle
自动压缩评估时报错, 最后map很低
使用下列命令评估时报错，评估出来map很低：
%cd /home/aistudio/work/PaddleSlim-develop/demo/auto_compression/detection/
!python run.py --config_path=./configs/ppyoloe_l_qat_dis.yaml --eval=True
报错为：Exception in thread Thread-1:
Traceback (most recent call last):
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 505, in _thread_loop
batch = self._get_data()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 634, in _get_data
batch.reraise()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/worker.py", line 169, in reraise
raise self.exc_type(msg)
ValueError: DataLoader worker(2) caught ValueError with message:
Traceback (most recent call last):
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/worker.py", line 329, in _worker_loop
batch = fetcher.fetch(indices)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/fetcher.py", line 134, in fetch
data = self.collate_fn(data)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddledet-2.3.0-py3.7.egg/ppdet/data/reader.py", line 91, in __call__
batch_data = default_collate_fn(data)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/collate.py", line 70, in default_collate_fn
for key in sample
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/collate.py", line 70, in <dictcomp>
for key in sample
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/collate.py", line 58, in default_collate_fn
batch = np.stack(batch, axis=0)
File "<__array_function__ internals>", line 6, in stack
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/numpy/core/shape_base.py", line 427, in stack
raise ValueError('all input arrays must have the same shape')
ValueError: all input arrays must have the same shape
@Sxx12 请问你的模型是怎么导出到预测模型的？没有复现你的这个问题，试过官方文档表格中的模型吗：https://bj.bcebos.com/v1/paddle-slim-models/detection/ppyoloe_crn_l_300e_coco.tar
@yghstill使用以下命令导出 %cd /home/aistudio/work/PaddleDetection !python tools/export_model.py -c configs/ppyoloe/ppyoloe_crn_l_300e_coco.yml -o weights=output/ppyoloe_crn_l_300e_coco/best_model_270.pdparams TestReader.fuse_normalize=true
使用的自己的数据集： metric: COCO num_classes: 7 TrainDataset: !COCODataSet image_dir: train anno_path: annotations/train.json dataset_dir: /home/aistudio/work/PaddleDetection/dataset/data_coco data_fields: ['image', 'gt_bbox', 'gt_class', 'is_crowd']
EvalDataset: !COCODataSet image_dir: eval anno_path: annotations/eval.json dataset_dir: /home/aistudio/work/PaddleDetection/dataset/data_coco
TestDataset: !ImageFolder anno_path: annotations/eval.json
改了模型路径其他基本按照配置文件，但是不论是压缩命令还是评估命令都会出现上述报错，不知道为什么，另外调用自动压缩命令时还会出现以下报错
Done (t=0.18s)
creating index...
index created!
[05/23 15:27:05] ppdet.data.source.coco WARNING: Found an invalid bbox in annotations: im_id: 6048, area: 0.0 x1: 1279, y1: 325, x2: 1280, y2: 325.
[05/23 15:27:05] ppdet.data.source.coco WARNING: Found an invalid bbox in annotations: im_id: 14216, area: 0.0 x1: 623, y1: 349, x2: 624, y2: 349.
2022-05-23 15:27:31,635-INFO: train config.distill_node_pair: ['teacher_concat_15.tmp_0', 'concat_15.tmp_0', 'teacher_concat_14.tmp_0', 'concat_14.tmp_0']
2022-05-23 15:27:32,945-INFO: quant_aware config {'weight_quantize_type': 'channel_wise_abs_max', 'activation_quantize_type': 'range_abs_max', 'weight_bits': 8, 'activation_bits': 8, 'not_quant_pattern': ['skip_quant'], 'quantize_op_types': ['conv2d', 'depthwise_conv2d'], 'dtype': 'int8', 'window_size': 10000, 'moving_rate': 0.9, 'for_tensorrt': False, 'is_full_quantize': False, 'use_pact': True, 'distill_loss': 'l2_loss', 'distill_node_pair': ['teacher_concat_15.tmp_0', 'concat_15.tmp_0', 'teacher_concat_14.tmp_0', 'concat_14.tmp_0'], 'teacher_model_dir': '/home/aistudio/work/PaddleDetection/output_inference/ppyoloe_crn_l_300e_coco/', 'merge_feed': True, 'teacher_params_filename': 'model.pdiparams', 'distill_lambda': 1.0, 'teacher_model_filename': 'model.pdmodel'}
2022-05-23 15:27:42,378-INFO: quant_aware config {'weight_quantize_type': 'channel_wise_abs_max', 'activation_quantize_type': 'range_abs_max', 'weight_bits': 8, 'activation_bits': 8, 'not_quant_pattern': ['skip_quant'], 'quantize_op_types': ['conv2d', 'depthwise_conv2d'], 'dtype': 'int8', 'window_size': 10000, 'moving_rate': 0.9, 'for_tensorrt': False, 'is_full_quantize': False, 'use_pact': True, 'distill_loss': 'l2_loss', 'distill_node_pair': ['teacher_concat_15.tmp_0', 'concat_15.tmp_0', 'teacher_concat_14.tmp_0', 'concat_14.tmp_0'], 'teacher_model_dir': '/home/aistudio/work/PaddleDetection/output_inference/ppyoloe_crn_l_300e_coco/', 'merge_feed': True, 'teacher_params_filename': 'model.pdiparams', 'distill_lambda': 1.0, 'teacher_model_filename': 'model.pdmodel'}
2022-05-23 15:35:48,122-INFO: When a preprocess_func is used in quant_aware, Need to save a mapping table to match variable names in the convert phase.
2022-05-23 15:35:48,122-INFO: The mapping table is saved as './mapping_table_for_saving_inference_model'.
Traceback (most recent call last):
File "run.py", line 191, in
@Sxx12 请问你的模型是怎么导出到预测模型的？没有复现你的这个问题，试过官方文档表格中的模型吗：https://bj.bcebos.com/v1/paddle-slim-models/detection/ppyoloe_crn_l_300e_coco.tar
你好，我也是一样的问题，我是在run的时候就报错。ppyoloe_l和ppyolo_s都会报一样的错误。麻烦复现排查一下。
paddle版本：paddle2.4rc0
paddleslim版本，PaddleSlim-develop
