TDAN-VSR-CVPR-2020 icon indicating copy to clipboard operation
TDAN-VSR-CVPR-2020 copied to clipboard

How to visualize the sampling positions?

Open pumpkinnan97 opened this issue 4 years ago • 5 comments

Hello, your work is so great! I am interested in your implementation details about the visualization of the sampling position in Fig.5 and Fig.6 of your paper. Could you please tell me more about the implementation details about it? I can't find the code in this repo. Thanks!

pumpkinnan97 avatar Sep 16 '20 09:09 pumpkinnan97

Thanks for your interest in our work. Previously, I printed the predict offsets and visualized the receptive fields accordingly. Read the model.py L2O2-206, we can know that sampling positions are decided by the 3x3 kernels and two predicted offsets which are used to sample support frames.

For one pixel in the predicted LR frame, it is corresponding to 9 pixels in aligned_fea. Then, for a pixel in aligned_fea, it is corresponding to 9 pixels (actual number should be 9^8 since we use the dconv grounp=8. In practice, a lot of pixels might be the same and we just use the most far away pixels for each axis) in fea. Furthermore, we can find associated positions in supp. In this way, we can visualize all 9^3 positions. But note the actual receptive field with propagated to image-level should be even larger since we have feature extraction module before alignment.

YapengTian avatar Sep 18 '20 02:09 YapengTian

I want to run on windows!

xiaomingxige avatar Sep 19 '20 00:09 xiaomingxige

Thanks for your interest in our work. Previously, I printed the predict offsets and visualized the receptive fields accordingly. Read the model.py L2O2-206, we can know that sampling positions are decided by the 3x3 kernels and two predicted offsets which are used to sample support frames.

For one pixel in the predicted LR frame, it is corresponding to 9 pixels in aligned_fea. Then, for a pixel in aligned_fea, it is corresponding to 9 pixels (actual number should be 9^8 since we use the dconv grounp=8. In practice, a lot of pixels might be the same and we just use the most far away pixels for each axis) in fea. Furthermore, we can find associated positions in supp. In this way, we can visualize all 9^3 positions. But note the actual receptive field with propagated to image-level should be even larger since we have feature extraction module before alignment.

Thanks for your reply! Now I know that you print the dconv offsets in the image. I understand how to implement it in theory. But I still can't understand how to implement it in code. If that's convenient, can you show me the code of this part? I'm really very interested in your work, and your paper looks great so that I want to obtain more details, thanks again for your code in this repo and helping!

pumpkinnan97 avatar Sep 22 '20 07:09 pumpkinnan97

Where did you see the visualization of the sampling position? I also want to see that. Where can I get the supplementary? @YapengTian @pumpkinnan97

sunnyHelen avatar Mar 29 '21 12:03 sunnyHelen

Please check the CVPR VERSION: https://openaccess.thecvf.com/content_CVPR_2020/papers/Tian_TDAN_Temporally-Deformable_Alignment_Network_for_Video_Super-Resolution_CVPR_2020_paper.pdf

YapengTian avatar Apr 02 '21 17:04 YapengTian