使用grounding-dino微调好的模型进行视频推理时报错AttributeError: 'DetDataSample' object has no attribute 'text'

[jamesbondzhou]

您好，我用grounding-dino微调自己的数据集得到的模型进行视频推理（demo/video_demo.py）时报错了，问题如下 (mmenv_new) (base) [zhoujianbang@ai mmdetection-dev-3.x]$ /ssd2/zhoujianbang/envs/mmenv_new/bin/python /ssd/zhoujianbang/project/mmdetection-dev-3.x/demo/video_demo.py Loads checkpoint by local backend from path: /ssd/zhoujianbang/project/mmdetection-dev-3.x/epoch_48.pth /ssd2/zhoujianbang/envs/mmenv_new/lib/python3.8/site-packages/mmengine/visualization/visualizer.py:196: UserWarning: Failed to add <class 'mmengine.visualization.vis_backend.LocalVisBackend'>, please provide the save_dir argument. warnings.warn(f'Failed to add {vis_backend.class}, ' Traceback (most recent call last): File "/ssd/zhoujianbang/project/mmdetection-dev-3.x/demo/video_demo.py", line 85, in main() File "/ssd/zhoujianbang/project/mmdetection-dev-3.x/demo/video_demo.py", line 63, in main result = inference_detector(model, frame, test_pipeline=test_pipeline) File "/ssd/zhoujianbang/project/mmdetection-main/mmdet/apis/inference.py", line 189, in inference_detector results = model.test_step(data_)[0] File "/ssd2/zhoujianbang/envs/mmenv_new/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 145, in test_step return self._run_forward(data, mode='predict') # type: ignore File "/ssd2/zhoujianbang/envs/mmenv_new/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 346, in _run_forward results = self(**data, mode=mode) File "/ssd2/zhoujianbang/envs/mmenv_new/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(*input, **kwargs) File "/ssd/zhoujianbang/project/mmdetection-main/mmdet/models/detectors/base.py", line 94, in forward return self.predict(inputs, data_samples) File "/ssd/zhoujianbang/project/mmdetection-main/mmdet/models/detectors/grounding_dino.py", line 509, in predict text_prompts.append(data_samples.text) AttributeError: 'DetDataSample' object has no attribute 'text'

这个模型使用image_demo.py可以正常推理，不过我看图片推理那里是有texts这个参数给入的，视频推理没有。 另外我查看之前的issue里有人提到应该使用dev-3.x这个分支的代码，可是还是一样的报错 模型的配置文件用的是configs/grounding_dino/grounding_dino_swin-t_finetune_16xb2_1x_coco.py

[simranbajaj06]

Did you find its solution @jamesbondzhou

[gong782008371]

我好像解决了，根据image_demo是缺少了text输入，所以查看了源码并进行手动提供了相应的text的值，具体修改： result = inference_detector(model, frame, test_pipeline=test_pipeline, text_prompt=text_prompt) 其中，text_prompt是我自定义的数据集的类别标签，其形式为[class1, class2, ...] 目前在我自己的数据集上能正常执行

[xuans29]

您好，我微调groundingdino的代码一直跑不起来，请问一下bert下载到本地的话代码要如何修改，另外能否看看我的issue中的步骤是否有误，非常感谢 https://github.com/open-mmlab/mmdetection/issues/11974