few-shot-object-detection
few-shot-object-detection copied to clipboard
ucbdrive
fine-tune result is great but demo can't detected anything
Hi I used a custom dataset on this project, which has great result in the fine tune step( eval result was great ) , but when I try to use the demo to detect the image, it does not detect anything, what is my problem?....base model, combine model and novel model can detect but final model from fine tune step can't!!!...(my dataset type was like pascal voc, jpg with xml).
i tested with both way like base train >> random initial >> fine tune and also base train >> random initial>> novel train >> combine and fine tune...all of them have good result in terminal but final model can't detected anything with demo ...this is my final result from fine tune stage:
thanks for help.
What command are you running when using the demo? Seems like an issue on the demo side rather than the model.
What command are you running when using the demo? Seems like an issue on the demo side rather than the model.
First thanks for your answer, I tried all week and could not solve the problem and pls support us until the complete solution pls.... well ...i used this command:
python3 -m demo.demo --config-file configs/PascalVOC-detection/split1/faster_rcnn_R_101_FPN_ft_all1_1shot.yaml --input image1.jpg --opts MODEL.WEIGHTS checkpoints/voc/faster_rcnn/faster_rcnn_R_101_FPN_ft_normalized_all1_1shot_randnovel/model_final.pth
this model_final was generated from fine tuning stage and my config is faster_rcnn_R_101_FPN_ft_all1_1shot.yaml and inside the config:
_BASE_: "../../Base-RCNN-FPN.yaml"
MODEL:
WEIGHTS: "checkpoints/voc/faster_rcnn/faster_rcnn_R_101_FPN_all2/model_reset_combine.pth"
MASK_ON: False
RESNETS:
DEPTH: 101
ROI_HEADS:
NUM_CLASSES: 32
OUTPUT_LAYER: "CosineSimOutputLayers"
FREEZE_FEAT: True
BACKBONE:
FREEZE: True
PROPOSAL_GENERATOR:
FREEZE: True
INPUT:
MIN_SIZE_TRAIN: (480, 512, 544, 576, 608, 640, 672, 704, 736, 768, 800)
MIN_SIZE_TEST: 800
DATASETS:
TRAIN: ('voc_2007_trainval_all1_1shot',)
TEST: ('voc_2007_test_all1',)
SOLVER:
IMS_PER_BATCH: 2
BASE_LR: 0.001
STEPS: (3000,)
MAX_ITER: 4000
CHECKPOINT_PERIOD: 500
WARMUP_ITERS: 0
OUTPUT_DIR: "checkpoints/voc/faster_rcnn/faster_rcnn_R_101_FPN_ft_normalized_all1_1shot_randnovel"
i have 32 class if i use novel train i need to choose my combine model for weights in fine tuning and if i use random intial i choose model surgery for weights but at all demo can not detected after fine tuning. I will rewrite the way I do my work in detail from the beginning. Maybe you can better guide me. first my dataset is like pascal voc, jpg for images and xml for labels. i use this command for base training :
CUDA_VISIBLE_DEVICES=0 python3 -m tools.train_net --config-file configs/PascalVOC-detection/split1/faster_rcnn_R_101_FPN_base1.yaml
and inside the base config is:
_BASE_: "../../Base-RCNN-FPN.yaml"
MODEL:
WEIGHTS: "/home/cudaa/Downloads/R-101.pkl"
MASK_ON: False
RESNETS:
DEPTH: 101
ROI_HEADS:
NUM_CLASSES: 24
INPUT:
MIN_SIZE_TRAIN: (480, 512, 544, 576, 608, 640, 672, 704, 736, 768, 800)
MIN_SIZE_TEST: 800
DATASETS:
TRAIN: ('voc_2007_trainval_base1',)
TEST: ('voc_2007_test_base1',)
SOLVER:
STEPS: (12000, 16000)
MAX_ITER: 18000 # 17.4 epochs
WARMUP_ITERS: 100
OUTPUT_DIR: "checkpoints/voc/faster_rcnn/faster_rcnn_R_101_FPN_base-final"
and because i have 32 class , 24 class for base and 8 class for novel...
after random initialize step i use this config faster_rcnn_R_101_FPN_ft_all1_1shot.yaml for fine tuning on surgery model (or use this command with 10shot) as i said above, and use this also for demo. Why my result in terminal was good but my model after fine tuning stage could not detected anything with demo?!
Thanks for the detailed explanation. I'm actually asking for the command you used when running the demo.py. My suspicion of the issue is the demo, since your evaluation results are good. Usually, not being able to detect anything means the model weights aren't loaded correctly.
Thanks for the detailed explanation. I'm actually asking for the command you used when running the
demo.py. My suspicion of the issue is the demo, since your evaluation results are good. Usually, not being able to detect anything means the model weights aren't loaded correctly.
What is your suggestion to solve this problem?
Oh, sorry I didn't see you already added the command you ran for the demo. It looks good to me, so I'll need to reproduce your issue on my side. Can you send me information about your environment? Python and packages versions that you used to run the code.
Oh, sorry I didn't see you already added the command you ran for the demo. It looks good to me, so I'll need to reproduce your issue on my side. Can you send me information about your environment? Python and packages versions that you used to run the code.
Sure, here you are:
Python 3.6.13 |Anaconda, Inc.| (default, Jun 4 2021, 14:25:59)
[GCC 7.5.0] on linux
absl-py==0.13.0
cachetools==4.2.2
certifi==2021.5.30
charset-normalizer==2.0.4
cloudpickle==1.6.0
cycler==0.10.0
Cython==0.29.24
dataclasses==0.8
detectron2==0.2.1+cu102
future==0.18.2
fvcore==0.1.5.post20210812
google-auth==1.34.0
google-auth-oauthlib==0.4.5
grpcio==1.39.0
idna==3.2
imagesize==1.2.0
importlib-metadata==4.6.4
iopath==0.1.9
kiwisolver==1.3.1
lvis==0.5.3
Markdown==3.3.4
matplotlib==3.3.4
mkl-fft==1.3.0
mkl-random==1.1.1
mkl-service==2.3.0
mock==4.0.3
numpy @ file:///tmp/build/80754af9/numpy_and_numpy_base_1603487797006/work
oauthlib==3.1.1
olefile==0.46
opencv-python==4.5.3.56
Pillow @ file:///tmp/build/80754af9/pillow_1625649052827/work
portalocker==2.3.0
protobuf==3.17.3
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycocotools==2.0.2
pydot==1.4.2
pyparsing==2.4.7
python-dateutil==2.8.2
PyYAML==5.4.1
requests==2.26.0
requests-oauthlib==1.3.0
rsa==4.7.2
six @ file:///tmp/build/80754af9/six_1623709665295/work
tabulate==0.8.9
tensorboard==2.6.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.0
termcolor==1.1.0
torch==1.6.0
torchvision==0.7.0
tqdm==4.62.0
typing-extensions==3.10.0.0
urllib3==1.26.6
Werkzeug==2.0.1
yacs==0.1.8
zipp==3.5.0
I dont know what happening after fine tuning!!!...base model result in terminal is good and also demo can detected with base model but after fine tuning this is very strange that result in terminal is perfect but demo could not detected any instances!!!...at all if you need any other thing like my model or any ....i can upload everything for you because i worked a lot of time and need to be successful...really thanks for your help.
this is also my final log from base and final step (fine tune stage): log-base train.txt log-fine tune train.txt :
I'm still waiting for you Thomas, miss you much, just please come back with good news :)...I tried a lot but I still did not get any results !
Apologies for the delay. Given that the demo works on the base model, that means the demo and model inference has no issues. That means the model itself has some issues. Can you provide some more details?
- What images are you running the demo on? What classes do they contain (base or novel)? Have you tried many images and none of them work?
- Have you tried running the demo on images from the training/validation set? Your fine-tuned model should be able to detect objects in the training images at least.
- Can you provide the terminal output after you run the demo with the fine-tuned model?
Apologies for the delay. Given that the demo works on the base model, that means the demo and model inference has no issues. That means the model itself has some issues. Can you provide some more details?
- What images are you running the demo on? What classes do they contain (base or novel)? Have you tried many images and none of them work?
- Have you tried running the demo on images from the training/validation set? Your fine-tuned model should be able to detect objects in the training images at least.
- Can you provide the terminal output after you run the demo with the fine-tuned model?
Thanks for answering, I tested train image,val image and also test image on the demo and nothing detected (novel and base not detected)...0 instances!!!...i tried many image and none of them work. I know my fine-tuned model should be able to detect objects in the training images at least but nothing detected!!!
this is the demo command and the terminal output after running demo :
demo command :
cudaa@cudaa-All-Series:~/Desktop/fsod$ python3 -m demo.demo --config-file configs/PascalVOC-detection/split1/faster_rcnn_R_101_FPN_ft_all1_5shot.yaml --input 000091.jpg --opts MODEL.WEIGHTS lastmodel/model_final.pth
terminal output:
[10/27 15:18:04 detectron2]: Arguments: Namespace(confidence_threshold=0.5, config_file='configs/PascalVOC-detection/split1/faster_rcnn_R_101_FPN_ft_all1_5shot.yaml', input=['000091.jpg'], opts=['MODEL.WEIGHTS', 'lastmodel/model_final.pth'], output=None, video_input=None, webcam=False)
froze backbone parameters
froze proposal generator parameters
froze roi_box_head parameters
[10/27 15:18:07 fvcore.common.checkpoint]: [Checkpointer] Loading from lastmodel/model_final.pth ...
/home/cudaa/Desktop/fsod/fsdet/modeling/roi_heads/fast_rcnn.py:115: UserWarning: This overload of nonzero is deprecated:
nonzero()
Consider using one of the following signatures instead:
nonzero(*, bool as_tuple) (Triggered internally at /opt/conda/conda-bld/pytorch_1595629427286/work/torch/csrc/utils/python_arg_parser.cpp:766.)
filter_inds = filter_mask.nonzero()
[10/27 15:18:07 detectron2]: 000091.jpg: detected 0 instances in 0.44s
What should i do Thomas?
what is the problem
------------------ 原始邮件 ------------------ 发件人: "ucbdrive/few-shot-object-detection" @.>; 发送时间: 2022年2月9日(星期三) 下午2:33 @.>; @.@.>; 主题: Re: [ucbdrive/few-shot-object-detection] fine-tune result is great but demo can't detected anything (#135)
did you solve it? @AISoltani @WumingzheAI
— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.***>
what is the problem … ------------------ 原始邮件 ------------------ 发件人: "ucbdrive/few-shot-object-detection" @.>; 发送时间: 2022年2月9日(星期三) 下午2:33 @.>; @.@.>; 主题: Re: [ucbdrive/few-shot-object-detection] fine-tune result is great but demo can't detected anything (#135) did you solve it? @AISoltani @WumingzheAI — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.***>
sorry,I @ wrong person, the problem same as fine-tune result is great but demo can't detected anything
你用的什么权重
---Original--- From: "Liu @.> Date: Thu, Feb 10, 2022 08:59 AM To: @.>; Cc: @.@.>; Subject: Re: [ucbdrive/few-shot-object-detection] fine-tune result is great but demo can't detected anything (#135)
what is the problem … ------------------ 原始邮件 ------------------ 发件人: "ucbdrive/few-shot-object-detection" @.>; 发送时间: 2022年2月9日(星期三) 下午2:33 @.>; @.@.>; 主题: Re: [ucbdrive/few-shot-object-detection] fine-tune result is great but demo can't detected anything (#135) did you solve it? @AISoltani @WumingzheAI — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.***>
sorry,I @ wrong person, the problem same as fine-tune result is great but demo can't detected anything
— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.***>
@AISoltani I need your help,I want to know did you solve the problem.I can detect base object after base training,but can not detect everything after finetune.
你用的什么权重 … ---Original--- From: "Liu @.> Date: Thu, Feb 10, 2022 08:59 AM To: @.>; Cc: @.@.>; Subject: Re: [ucbdrive/few-shot-object-detection] fine-tune result is great but demo can't detected anything (#135) what is the problem … ------------------ 原始邮件 ------------------ 发件人: "ucbdrive/few-shot-object-detection" @.>; 发送时间: 2022年2月9日(星期三) 下午2:33 @.>; @.@.>; 主题: Re: [ucbdrive/few-shot-object-detection] fine-tune result is great but demo can't detected anything (#135) did you solve it? @AISoltani @WumingzheAI — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.> sorry,I @ wrong person, the problem same as fine-tune result is great but demo can't detected anything — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.>
我是base9类，novel3类，base类训练完的模型可以正常检测base9类，fine tune完之后的模型连9类都检测不到，所有的12类也检测不到。但是训练log里面eval的结果还可以。
你用的什么权重 … ---Original--- From: "Liu @.> Date: Thu, Feb 10, 2022 08:59 AM To: _@**._>; Cc: _@.@.>; Subject: Re: [ucbdrive/few-shot-object-detection] fine-tune result is great but demo can't detected anything (#135) what is the problem … ------------------ 原始邮件 ------------------ 发件人: "ucbdrive/few-shot-object-detection" @.>; 发送时间: 2022年2月9日(星期三) 下午2:33 @.>; @.@.>; 主题: Re: [ucbdrive/few-shot-object-detection] fine-tune result is great but demo can't detected anything (#135) did you solve it? @AISoltani @WumingzheAI — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.> sorry,I @ wrong person, the problem same as fine-tune result is great but demo can't detected anything — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.**_>
我是base9类，novel3类，base类训练完的模型可以正常检测base9类，fine tune完之后的模型连9类都检测不到，所有的12类也检测不到。但是训练log里面eval的结果还可以。
请问问题解决了吗 我也是只能检测到base类，novel类都检测不到
