PaddleDetection RT-DETR官方代码已发布

论文: DETRs Beat YOLOs on Real-time Object Detection 代码:

https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/rtdetr
https://github.com/lyuwenyu/RT-DETR/

简介

RT-DETR是第一个实时端到端目标检测器。具体而言，我们设计了一个高效的混合编码器，通过解耦尺度内交互和跨尺度融合来高效处理多尺度特征，并提出了IoU感知的查询选择机制，以优化解码器查询的初始化。此外，RT-DETR支持通过使用不同的解码器层来灵活调整推理速度，而不需要重新训练，这有助于实时目标检测器的实际应用。RT-DETR-L在COCO val2017上实现了53.0%的AP，在T4 GPU上实现了114FPS，RT-DETR-X实现了54.8%的AP和74FPS，在速度和精度方面都优于相同规模的所有YOLO检测器。RT-DETR-R50实现了53.1%的AP和108FPS，RT-DETR-R101实现了54.3%的AP和74FPS，在精度上超过了全部使用相同骨干网络的DETR检测器。若要了解更多细节，请参考我们的论文paper.

Apr 20 '23 01:04 lyuwenyu

如何训练自己的数据集呢

https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/data/PrepareDetDataSet.md

Apr 21 '23 02:04 lyuwenyu

请问有没有对比过INT8下的速度表现呢

Apr 21 '23 03:04 LeoCho1learning

请问有没有对比过INT8下的速度表现呢

已经在做了，( 如果用INT8 RT-DETR可能优势会更明显 ( 总的时间缩短但是带后处理的模型后处理时间是稳定的

Apr 21 '23 03:04 lyuwenyu

不能eval,只能训练 (pytorch) C:\Users\lenovo\Desktop\study\PaddleDetection-develop\PaddleDetection-develop>python -u tools/eval.py -c configs/rtdetr/rtdetr_r50vd_6x_coco.yml -o weights=output/rtdetr_r50vd_6x_coco/8.pdparams Warning: Unable to use JDE/FairMOT/ByteTrack, please install lap, for example: pip install lap, see https://github.com/gatagat/lap Warning: import ppdet from source directory without installing, run 'python setup.py install' to install ppdet firstly W0421 15:32:24.823109 15072 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 8.6, Driver API Version: 11.5, Runtime API Version: 11.2 W0421 15:32:24.826112 15072 gpu_resources.cc:91] device: 0, cuDNN Version: 8.3. loading annotations into memory... Done (t=0.03s) creating index... index created! [04/21 15:32:26] ppdet.data.source.coco INFO: Load [600 samples valid, 0 samples invalid] in file dataset/coco\annotations/instances_train2017.json. [04/21 15:32:27] ppdet.utils.checkpoint INFO: Finish loading model weights: output/rtdetr_r50vd_6x_coco/8.pdparams Traceback (most recent call last): File "tools/eval.py", line 208, in main() File "tools/eval.py", line 204, in main run(FLAGS, cfg) File "tools/eval.py", line 159, in run trainer.evaluate() File "C:\Users\lenovo\Desktop\study\PaddleDetection-develop\PaddleDetection-develop\ppdet\engine\trainer.py", line 672, in evaluate self._eval_with_loader(self.loader) File "C:\Users\lenovo\Desktop\study\PaddleDetection-develop\PaddleDetection-develop\ppdet\engine\trainer.py", line 641, in _eval_with_loader metric.update(data, outs) File "C:\Users\lenovo\Desktop\study\PaddleDetection-develop\PaddleDetection-develop\ppdet\metrics\metrics.py", line 114, in update infer_results = get_infer_results( File "C:\Users\lenovo\Desktop\study\PaddleDetection-develop\PaddleDetection-develop\ppdet\metrics\coco_utils.py", line 52, in get_infer_results infer_res['bbox'] = get_det_res( File "C:\Users\lenovo\Desktop\study\PaddleDetection-develop\PaddleDetection-develop\ppdet\metrics\json_results.py", line 30, in get_det_res category_id = label_to_cat_id_map[int(num_id)] KeyError: 1

Apr 21 '23 07:04 1117wwh

训练的自己的数据吧, 应该是val.json里的类别不全， category_id = label_to_cat_id_map[int(num_id)]换成category_id = int(num_id)试一下，，或者参考这个issue https://github.com/PaddlePaddle/PaddleDetection/issues/4777

Apr 21 '23 08:04 lyuwenyu

请问如何对转出的onnx和trt进行推理呢谢谢

Apr 21 '23 08:04 itsrealkk

看代码发现HybridEncoder里连MSDeformableAttention都去掉了？好厉害。

Apr 21 '23 08:04 TsingWei

请问如何对转出的onnx和trt进行推理呢谢谢

在readme里有具体操作的步骤

Apr 21 '23 09:04 lyuwenyu

请问 File "D:\Code\LJJ\rtdetr\PaddleDetection\ppdet\core\workspace.py", line 229, in create raise ValueError("The module {} is not registered".format(name)) ValueError: The module RTDETRTransformer is not registered 这是什么原因呢？

Apr 22 '23 11:04 jingjingli066

预训练模型的下载路径，文档的描述和配置文件里的默认值不一致。 hgnet_v2用2个预训练模型都提示不匹配；提示信息in model is not matched with pretrained weights, and its will be trained from scratch r50vd可以进入训练，在log出第1个iter信息后卡死。机器是windows环境，rtx3090GPU

Apr 23 '23 04:04 shengs1113

PP-YOLOE+_l 的 mAP 是 52点多，yolov5_l是67.3，论文的图中为什么mAP那么低？

Apr 23 '23 14:04 wm901115nwpu

请问 File "D:\Code\LJJ\rtdetr\PaddleDetection\ppdet\core\workspace.py", line 229, in create raise ValueError("The module {} is not registered".format(name)) ValueError: The module RTDETRTransformer is not registered 这是什么原因呢？

pull最新的代码试一下，我们这边测试没有出现你说的这个情况

Apr 24 '23 02:04 lyuwenyu

请问有添加到FastDeploy的计划么？

Apr 24 '23 11:04 hhxdestiny

PP-YOLOE+_l 的 mAP 是 52点多，yolov5_l是67.3，论文的图中为什么mAP那么低？

看看论文，你看窜列了，你看的是红色这一列

Apr 25 '23 01:04 hhxdestiny

好的

Apr 25 '23 02:04 wm901115nwpu

请问有添加到FastDeploy的计划么？

已经在做了 @hhxdestiny

Apr 25 '23 02:04 lyuwenyu

能给一个怎么使用onnx模型的演示代码吗？

直接用官方教程，会报错

❯ python tools/infer.py -c configs/rtdetr/rtdetr_r101vd_6x_coco.yml \
 -o weights=/root/art/paddle/model/rtdetr_r101vd_6x_coco.pdparams \
 --infer_img=/root/art/paddle/test/1.jpg

[04/25 16:34:22] ppdet.utils.checkpoint INFO: Finish loading model weights: /root/art/paddle/model/rtdetr_r101vd_6x_coco.pdparams
[04/25 16:34:22] ppdet.data.source.category WARNING: anno_file 'dataset/coco/annotations/instances_val2017.json' is None or not set or not exist, please recheck TrainDataset/EvalDataset/Test
Dataset.anno_path, otherwise the default categories will be used by metric_type.
[04/25 16:34:22] ppdet.data.source.category WARNING: metric_type: COCO, load default categories of COCO.
  0%|                                                                                                                                                                   | 0/1 [00:00<?, ?it/s]
[04/25 16:34:22] reader WARNING: fail to map sample transform [Decode_13aee1] with error: 'bytes' object has no attribute 'shape' and stack:
Traceback (most recent call last):
  File "/root/art/paddle/PaddleDetection/ppdet/data/reader.py", line 59, in __call__
    data = f(data)
  File "/root/art/paddle/PaddleDetection/ppdet/data/transform/operators.py", line 105, in __call__
    sample = self.apply(sample, context)
  File "/root/art/paddle/PaddleDetection/ppdet/data/transform/operators.py", line 138, in apply
    sample['h'] = im.shape[0]
AttributeError: 'bytes' object has no attribute 'shape'

Exception in thread Thread-1 (_thread_loop):
Traceback (most recent call last):
  File "/root/.local/share/rtx/installs/python/3.10.11/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/root/.local/share/rtx/installs/python/3.10.11/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/root/art/paddle/.direnv/python-3.10.11/lib/python3.10/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 247, in _thread_loop
    batch = self._dataset_fetcher.fetch(
  File "/root/art/paddle/.direnv/python-3.10.11/lib/python3.10/site-packages/paddle/fluid/dataloader/fetcher.py", line 127, in fetch
    data.append(self.dataset[idx])
  File "/root/art/paddle/PaddleDetection/ppdet/data/source/dataset.py", line 102, in __getitem__
    return self.transform(roidb)
  File "/root/art/paddle/PaddleDetection/ppdet/data/reader.py", line 65, in __call__
    raise e
  File "/root/art/paddle/PaddleDetection/ppdet/data/reader.py", line 59, in __call__
    data = f(data)
  File "/root/art/paddle/PaddleDetection/ppdet/data/transform/operators.py", line 105, in __call__
    sample = self.apply(sample, context)
  File "/root/art/paddle/PaddleDetection/ppdet/data/transform/operators.py", line 138, in apply
    sample['h'] = im.shape[0]
AttributeError: 'bytes' object has no attribute 'shape'

用官方的 rtdetr_r101vd_6x_coco.onnx 和这篇文章中的代码 https://zhuanlan.zhihu.com/p/622940435

运行还是报错

Traceback (most recent call last):

File "/Users/z/git/paddle/./test.py", line 48, in

output = session.run(None, {'image': img})

File "/Users/z/.local/share/rtx/installs/python/3.10.11/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 196, in run

raise ValueError("Model requires {} inputs. Input Feed contains {}".format(num_required_inputs, num_inputs))

ValueError: Model requires 3 inputs. Input Feed contains 1

改成 https://zhuanlan.zhihu.com/p/622940435 中的的模型才能运行

Apr 25 '23 03:04 wacdev

我可以正常eval，但在训练中遇到这样一个问题，rtx2080 8g

/home/sc/.local/lib/python3.7/site-packages/matplotlib-3.4.0rc1-py3.7-linux-x86_64.egg/matplotlib/init.py:152: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. if LooseVersion(module.version) < minver: /home/sc/development/anaconda3/envs/rtdetr/lib/python3.7/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. other = LooseVersion(other) Warning: Unable to use JDE/FairMOT/ByteTrack, please install lap, for example: pip install lap, see https://github.com/gatagat/lap Warning: Unable to use numba in PP-Tracking, please install numba, for example(python3.7): pip install numba==0.56.4 Warning: Unable to use numba in PP-Tracking, please install numba, for example(python3.7): pip install numba==0.56.4 Warning: Unable to use MOT metric, please install motmetrics, for example: pip install motmetrics, see https://github.com/longcw/py-motmetrics Warning: Unable to use MCMOT metric, please install motmetrics, for example: pip install motmetrics, see https://github.com/longcw/py-motmetrics Warning: import ppdet from source directory without installing, run 'python setup.py install' to install ppdet firstly loading annotations into memory... Done (t=9.65s) creating index... index created! [04/25 14:31:57] ppdet.data.source.coco WARNING: Found an invalid bbox in annotations: im_id: 200365, area: 0.0 x1: 296.65, y1: 388.33, x2: 297.67999999999995, y2: 388.33. [04/25 14:32:08] ppdet.data.source.coco WARNING: Found an invalid bbox in annotations: im_id: 550395, area: 0.0 x1: 9.98, y1: 188.56, x2: 15.52, y2: 188.56. [04/25 14:32:09] ppdet.data.source.coco INFO: Load [117266 samples valid, 1021 samples invalid] in file /media/sc/mSc/wkp/2017/annotations/instances_train2017.json. W0425 14:32:10.895720 2548 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.5, Driver API Version: 11.4, Runtime API Version: 10.2 W0425 14:32:10.896754 2548 gpu_resources.cc:91] device: 0, cuDNN Version: 8.7. [04/25 14:32:13] ppdet.utils.checkpoint INFO: ['fc.bias', 'fc.weight', 'last_conv.weight'] in pretrained weight is not used in the model, and its will not be loaded [04/25 14:32:13] ppdet.utils.checkpoint INFO: Finish loading model weights: /home/sc/.cache/paddle/weights/PPHGNetV2_X_ssld_pretrained.pdparams

C++ Traceback (most recent call last):

0 paddle::pybind::ThrowExceptionToPython(std::__exception_ptr::exception_ptr)

Error Message Summary:

FatalError: Process abort signal is detected by the operating system. [TimeInfo: *** Aborted at 1682404354 (unix time) try "date -d @1682404354" if you are using GNU date ***] [SignalInfo: *** SIGABRT (@0x3e8000009f4) received by PID 2548 (TID 0x7f57b424f740) from PID 2548 ***]

/home/sc/code/rt-dert/train.sh: 行 2: 2548 已放弃 python PaddleDetection-develop/tools/train.py -c PaddleDetection-develop/confrtdetr/rtdetr_hgnetv2_x_6x_coco.yml --eval

Apr 25 '23 07:04 Originlightwkp

请问有没有对比过INT8下的速度表现呢

已经在做了，( 如果用INT8 RT-DETR可能优势会更明显 ( 总的时间缩短但是带后处理的模型后处理时间是稳定的

您好，反馈一个我们某家国产芯片编译器出现的情况。麻烦大佬也能综合一下，考虑下我们部署环境的局限性，十分感谢。出现了部分算子不支持的情况：Shape、Cast、Where、range、Topk、Tile、GatherND、GridSample、Mod，还有Slice算子，目前编译器仅支持通道维度上的切分。

May 04 '23 07:05 sdreamforchen

您好，有计划发布pytorch版本吗

May 06 '23 10:05 eorroot

隔壁 ultralytics 已经在做 pytorch 版本了

May 06 '23 13:05 zhiqwang

用RTDETR进行SAHI切图拼图评估时报错

项目“RTDETR+SAHI切图”共享链接(有效期三天)：https://aistudio.baidu.com/studio/project/partial/verify/6092017/b32e0f8533b2482aa1675a99aa50aeec

/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/init.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import MutableMapping /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Iterable, Mapping /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Sized W0505 10:40:56.459722 895 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2 W0505 10:40:56.464571 895 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2. loading annotations into memory... Done (t=0.40s) creating index... index created! [05/05 10:41:09] ppdet.data.source.coco INFO: 548 samples and slice to 2842 sub_samples in file /home/aistudio/MyDataset/val.json [05/05 10:41:11] ppdet.utils.checkpoint INFO: Finish loading model weights: output/rtdetr_r101vd_6x_coco/best_model.pdparams Traceback (most recent call last): File "tools/eval.py", line 208, in main() File "tools/eval.py", line 204, in main run(FLAGS, cfg) File "tools/eval.py", line 157, in run match_metric=FLAGS.match_metric) File "/home/aistudio/PaddleDetection/ppdet/engine/trainer.py", line 764, in evaluate_slice match_metric) File "/home/aistudio/PaddleDetection/ppdet/engine/trainer.py", line 708, in _eval_with_loader_slice outs['bbox'][:, 2:4] = outs['bbox'][:, 2:4] + shift_amount File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/math_op_patch.py", line 304, in impl return math_op(self, other_var, -1) ValueError: (InvalidArgument) Broadcast dimension mismatch. Operands could not be broadcast together with the shape of X = [1200, 2] and the shape of Y = [4, 2]. Received [1200] in X is not equal to [4] in Y at i:0. [Hint: Expected x_dims_array[i] == y_dims_array[i] || x_dims_array[i] <= 1 || y_dims_array[i] <= 1 == true, but received x_dims_array[i] == y_dims_array[i] || x_dims_array[i] <= 1 || y_dims_array[i] <= 1:0 != true:1.] (at /paddle/paddle/phi/kernels/funcs/common_shape.h:84)