frostinassiky
How to use PGCN to postprocess the results?
Hi, I use the 'detection.json' file in the output folder to generate a similar propfile with bsn_test_prop_file in PGCN. However, I got lower results than the original results of gtad itself. I don't know where I did wrong.
Does it works for you?
Postprocessing without conducting NMS might be help. Does it works for you?
Guys sorry for the delay! Here is the update. The G-TAD+PGCN experiment was implemented on another machine, but I have a chance to find the script that I used for the experiment. :100:
Some extra details: 1, G-TAD predictions that have low confidence scores (e.g. 0.1) are discarded. 2, Original P-GCN proposals are kept to maintain the distribution of the start/end times. This is important because the P-GCN model is already trained with its own proposal. 3, Some hyperparameters of P-GCN need to be adjusted accordingly, such as combination weights, nms threshold.
New code: Please add the following script in the dataset class (pgcn_dataset.py) of P-GCN code here.
################################### GTAD ########################################################
use_gtad = True
gtad_thres = 0.1
if use_gtad:
'''Load GTAD result'''
import pandas as pd
data = pd.read_csv('/home/xum/PGCN/data/detection_results.txt', sep=" ", header=None)
data.columns = ["vid", "s", "e", "cls", "score"]
vids = [x for x in set(data.vid.values.tolist()) if type(x) != float]
'''Load THUMOS infomations'''
thumos_info = pd.read_csv('/home/xum/PGCN/data/thumos_info.csv', sep=",")
thumos_cls = pd.read_csv('/home/xum/PGCN/data/detclasslist.txt', sep=" ", header=None)
thumos_cls_idx = {thumos_cls.iloc[x, 0]: x for x in range(20)}
'''Replace each proposal'''
for per_prop in prop_info:
vid = per_prop[0].split('/')[-1]
if not vid in vids:
print('video {} has zero proposal from gtad'.format(vid)) # check it exists
continue
gtad_vid = data[data.vid == vid] # unit is second
frate = thumos_info[thumos_info['video-name'] == vid]['frame-rate'].values[0]
starts = (gtad_vid.s * frate).values
ends = (gtad_vid.e * frate).values
scores = gtad_vid.score.values
clses = [thumos_cls_idx[x] for x in gtad_vid.cls.values]
for idx, (c, sc1, sc2, s, e) in enumerate(zip(clses, scores, scores, starts, ends)):
if sc1 > gtad_thres:
per_prop[3].insert(0, [str(int(c)), str(sc1), str(sc2), str(int(s)), str(int(e))])
################################################################################################################
Please let me know if you have any questions.
Guys sorry for the delay! Here is the update. The G-TAD+PGCN experiment was implemented on another machine, but I have a chance to find the script that I used for the experiment. 💯
Some extra details: 1, G-TAD predictions that have low confidence scores (e.g. 0.1) are discarded. 2, Original P-GCN proposals are kept to maintain the distribution of the start/end times. This is important because the P-GCN model is already trained with its own proposal. 3, Some hyperparameters of P-GCN need to be adjusted accordingly, such as combination weights, nms threshold.
New code: Please add the following script in the dataset class (pgcn_dataset.py) of P-GCN code here.
################################### GTAD ######################################################## use_gtad = True gtad_thres = 0.1 if use_gtad: '''Load GTAD result''' import pandas as pd data = pd.read_csv('/home/xum/PGCN/data/detection_results.txt', sep=" ", header=None) data.columns = ["vid", "s", "e", "cls", "score"] vids = [x for x in set(data.vid.values.tolist()) if type(x) != float] '''Load THUMOS infomations''' thumos_info = pd.read_csv('/home/xum/PGCN/data/thumos_info.csv', sep=",") thumos_cls = pd.read_csv('/home/xum/PGCN/data/detclasslist.txt', sep=" ", header=None) thumos_cls_idx = {thumos_cls.iloc[x, 0]: x for x in range(20)} '''Replace each proposal''' for per_prop in prop_info: vid = per_prop[0].split('/')[-1] if not vid in vids: print('video {} has zero proposal from gtad'.format(vid)) # check it exists continue gtad_vid = data[data.vid == vid] # unit is second frate = thumos_info[thumos_info['video-name'] == vid]['frame-rate'].values[0] starts = (gtad_vid.s * frate).values ends = (gtad_vid.e * frate).values scores = gtad_vid.score.values clses = [thumos_cls_idx[x] for x in gtad_vid.cls.values] for idx, (c, sc1, sc2, s, e) in enumerate(zip(clses, scores, scores, starts, ends)): if sc1 > gtad_thres: per_prop[3].insert(0, [str(int(c)), str(sc1), str(sc2), str(int(s)), str(int(e))]) ################################################################################################################Please let me know if you have any questions.
Thanks for your reply, but how can I get the "/home/xum/PGCN/data/detection_results.txt"?
Hi @RonanDou The "detection_results.txt" file is the txt format of the json file after post-processing.
You may need to change the line here.
https://github.com/frostinassiky/gtad/blob/6deb5b1bc6883b48bd22e0cc593069643c953e3d/gtad_postprocess.py#L154-L156
Hi @RonanDou The "detection_results.txt" file is the txt format of the json file after post-processing.
You may need to change the line here.
https://github.com/frostinassiky/gtad/blob/6deb5b1bc6883b48bd22e0cc593069643c953e3d/gtad_postprocess.py#L154-L156
I do the experiment according to what you describe, but I get lower results than the original results of gtad itself. Can you give a little more implementation detail? such as: hyperparameters. Thank you !!
Hi @RonanDou As I remember, I changed the NMS threshold and weights to combine RBG and optical flow results.
Since you find the P-GCN result is worse than G-TAD, what if you decrease gtad_thres to improve more low-confident predictions?
Hi @RonanDou The "detection_results.txt" file is the txt format of the json file after post-processing. You may need to change the line here. https://github.com/frostinassiky/gtad/blob/6deb5b1bc6883b48bd22e0cc593069643c953e3d/gtad_postprocess.py#L154-L156
I do the experiment according to what you describe, but I get lower results than the original results of gtad itself. Can you give a little more implementation detail? such as: hyperparameters. Thank you !!
Hi, would u tell me how to get this file 'thumos info.csv'? thanks a lot!
Hi, I use the 'detection.json' file in the output folder to generate a similar propfile with bsn_test_prop_file in PGCN. However, I got lower results than the original results of gtad itself. I don't know where I did wrong.
Hi, Could you share your code for generating the propfile with bsn_test_prop_file?
I meet the same problem, did you solve it? thanks u!!
Hi, Could you share your code for generating the propfile with bsn_test_prop_file?
Postprocessing without conducting NMS might be help.
Hi, Could you share your code for generating the propfile with bsn_test_prop_file?
Hi @RonanDou The "detection_results.txt" file is the txt format of the json file after post-processing. You may need to change the line here. https://github.com/frostinassiky/gtad/blob/6deb5b1bc6883b48bd22e0cc593069643c953e3d/gtad_postprocess.py#L154-L156
I do the experiment according to what you describe, but I get lower results than the original results of gtad itself. Can you give a little more implementation detail? such as: hyperparameters. Thank you !!
Hi, could you share how to get the 'detection_results.txt'? I try to save the content of json file as txt. But it can not be used by the provided code.
Hi ，Could you help me solve a few problems
- I get the ”detection_result.txt“ The format is as follows：
vid,s,e,cls,score
video_test_0000004,19.5,22.3,CricketBowling,0.323464
video_test_0000004,19.5,22.3,CricketShot,0.271474
video_test_0000004,0.0,0.5,CricketBowling,0.177645
video_test_0000004,0.0,0.5,CricketShot,0.149093
video_test_0000004,21.3,22.5,CricketBowling,0.156935
video_test_0000004,21.3,22.5,CricketShot,0.131711
...........
I get the “thumos_info.csv” ,The format is as follows：
video-name,t-init,t-end,f-init,f-end,video-duration,frame-rate,video-frames,label-idx
video_test_0000278,0.0,1.4,0,42,215.166667,30.0,6455,-1
video_test_0000278,95.7,97.2,2871,2916,215.166667,30.0,6455,-1
video_test_0000293,50.6,54.6,1518,1638,233.466667,30.0,7004,-1
video_test_0000293,67.4,71.7,2022,2151,233.466667,30.0,7004,-1
video_test_0000293,99.7,106.4,2991,3192,233.466667,30.0,7004,-1
video_test_0000293,118.1,126.4,3543,3792,233.466667,30.0,7004,-1
..........
the "detclasslist.txt" ,The format is as follows：
BaseballPitch
BasketballDunk
Billiards
CleanAndJerk
CliffDiving
CricketBowling
CricketShot
Diving
FrisbeeCatch
........
I did not add the above Gtad code in PGCN and run test_two_stream.sh
the result：
average map ：0.4266
I add the above Gtad code in PGCN and run test_two_stream.sh
the result：
average map ：0.4266
Why are the above two results the same ?
Could you tell me where I went wrong？Thanks
Hi ，Could you help me solve a few problems
- I get the ”detection_result.txt“ The format is as follows：
vid,s,e,cls,score video_test_0000004,19.5,22.3,CricketBowling,0.323464 video_test_0000004,19.5,22.3,CricketShot,0.271474 video_test_0000004,0.0,0.5,CricketBowling,0.177645 video_test_0000004,0.0,0.5,CricketShot,0.149093 video_test_0000004,21.3,22.5,CricketBowling,0.156935 video_test_0000004,21.3,22.5,CricketShot,0.131711 ...........I get the “thumos_info.csv” ,The format is as follows：
video-name,t-init,t-end,f-init,f-end,video-duration,frame-rate,video-frames,label-idx video_test_0000278,0.0,1.4,0,42,215.166667,30.0,6455,-1 video_test_0000278,95.7,97.2,2871,2916,215.166667,30.0,6455,-1 video_test_0000293,50.6,54.6,1518,1638,233.466667,30.0,7004,-1 video_test_0000293,67.4,71.7,2022,2151,233.466667,30.0,7004,-1 video_test_0000293,99.7,106.4,2991,3192,233.466667,30.0,7004,-1 video_test_0000293,118.1,126.4,3543,3792,233.466667,30.0,7004,-1 ..........the "detclasslist.txt" ,The format is as follows：
BaseballPitch BasketballDunk Billiards CleanAndJerk CliffDiving CricketBowling CricketShot Diving FrisbeeCatch ........I did not add the above Gtad code in PGCN and run
test_two_stream.shthe result： average map ：0.4266I add the above Gtad code in PGCN and run
test_two_stream.shthe result： average map ：0.4266 Why are the above two results the same ?Could you tell me where I went wrong？Thanks
Could you please share the detclasslist.txt and thumos_info.csv to me? Thanks
Hi ，Could you help me solve a few problems
- I get the ”detection_result.txt“ The format is as follows：
vid,s,e,cls,score video_test_0000004,19.5,22.3,CricketBowling,0.323464 video_test_0000004,19.5,22.3,CricketShot,0.271474 video_test_0000004,0.0,0.5,CricketBowling,0.177645 video_test_0000004,0.0,0.5,CricketShot,0.149093 video_test_0000004,21.3,22.5,CricketBowling,0.156935 video_test_0000004,21.3,22.5,CricketShot,0.131711 ...........I get the “thumos_info.csv” ,The format is as follows：
video-name,t-init,t-end,f-init,f-end,video-duration,frame-rate,video-frames,label-idx video_test_0000278,0.0,1.4,0,42,215.166667,30.0,6455,-1 video_test_0000278,95.7,97.2,2871,2916,215.166667,30.0,6455,-1 video_test_0000293,50.6,54.6,1518,1638,233.466667,30.0,7004,-1 video_test_0000293,67.4,71.7,2022,2151,233.466667,30.0,7004,-1 video_test_0000293,99.7,106.4,2991,3192,233.466667,30.0,7004,-1 video_test_0000293,118.1,126.4,3543,3792,233.466667,30.0,7004,-1 ..........the "detclasslist.txt" ,The format is as follows：
BaseballPitch BasketballDunk Billiards CleanAndJerk CliffDiving CricketBowling CricketShot Diving FrisbeeCatch ........I did not add the above Gtad code in PGCN and run
test_two_stream.shthe result： average map ：0.4266I add the above Gtad code in PGCN and run
test_two_stream.shthe result： average map ：0.4266 Why are the above two results the same ?Could you tell me where I went wrong？Thanks
Because the test_two_stream.sh file reads the results from already generated results/Flow_result results/RGB_result files
Hi ，Could you help me solve a few problems
- I get the ”detection_result.txt“ The format is as follows：
vid,s,e,cls,score video_test_0000004,19.5,22.3,CricketBowling,0.323464 video_test_0000004,19.5,22.3,CricketShot,0.271474 video_test_0000004,0.0,0.5,CricketBowling,0.177645 video_test_0000004,0.0,0.5,CricketShot,0.149093 video_test_0000004,21.3,22.5,CricketBowling,0.156935 video_test_0000004,21.3,22.5,CricketShot,0.131711 ...........I get the “thumos_info.csv” ,The format is as follows：
video-name,t-init,t-end,f-init,f-end,video-duration,frame-rate,video-frames,label-idx video_test_0000278,0.0,1.4,0,42,215.166667,30.0,6455,-1 video_test_0000278,95.7,97.2,2871,2916,215.166667,30.0,6455,-1 video_test_0000293,50.6,54.6,1518,1638,233.466667,30.0,7004,-1 video_test_0000293,67.4,71.7,2022,2151,233.466667,30.0,7004,-1 video_test_0000293,99.7,106.4,2991,3192,233.466667,30.0,7004,-1 video_test_0000293,118.1,126.4,3543,3792,233.466667,30.0,7004,-1 ..........the "detclasslist.txt" ,The format is as follows：
BaseballPitch BasketballDunk Billiards CleanAndJerk CliffDiving CricketBowling CricketShot Diving FrisbeeCatch ........I did not add the above Gtad code in PGCN and run
test_two_stream.shthe result： average map ：0.4266 I add the above Gtad code in PGCN and runtest_two_stream.shthe result： average map ：0.4266 Why are the above two results the same ? Could you tell me where I went wrong？ThanksCould you please share the detclasslist.txt and thumos_info.csv to me? Thanks You can contact me via my email [email protected]. I don’t know if the two files I generated are correct. Maybe we can discuss it in detail.