Performance issue and data generation

[CHELSEA234]

Hi, I think your implementation here is brilliant, but I have two questions about it:

1. I download model_50_epoch.pth and test it on Set5 data, the PSNR is 37.44 which isn't 37.65. I used bicubic interpolation in Matlab.

2. I use your data_generation file to create 92608 images, I trained VDSR with augmentation that increase image number up to 1111296. However, the performance PSNR is just 32.26.

Can you give me some advice here, I use python3.6 and latest version Pytorch

[twtygqyy]

@CHELSEA234 Thanks for your interests. Could you please check if the bicubic performance is close to the result on the paper? Also please save the bicubiced images in mat format or use matlab function in python. I imported matlab.engine in python and processed all matlab functions in python for best result.

In addition, please generate the test samples as follows and evaluate PSNR only on Y channel

im_gt = imread(img_path);
im_gt = modcrop(im_gt,scale);
im_gt = double(im_gt);
im_gt_ycbcr = rgb2ycbcr(im_gt / 255.0);
im_gt_y = im_gt_ycbcr(:,:,1) * 255.0;
im_l_ycbcr = imresize(im_gt_ycbcr,1/scale,'bicubic');
im_b_ycbcr = imresize(im_l_ycbcr,scale,'bicubic');
im_b_y = im_b_ycbcr(:,:,1) * 255.0;
im_b = ycbcr2rgb(im_b_ycbcr) * 255.0;

[CHELSEA234]

@twtygqyy Thanks for reply. How did you do data augmentation about raw_data? Rotation and flipping?

====================================================== I follow your code to get expected result, thanks! This also proves that my ways to calculate PSRN and SSIM are right. Did you just input 291 images into data_generation code to get about 29000 images without any augmentation? 27th July

====================================================== I redo training data and store them in train.h5, then import train.h5 into python. It gives me almost same PSNR(37.45 for upscaling factor x2).

I used mat format for testing image, and h5py file as training data. Should I change your data generation code a little to include data type conversion like what you wrote in test samples generation?

Can you tell me which part you use matlab.engine in python? I don't see this in your code, including LapSR.

I will be very grateful to your help...

28th July

[twtygqyy]

@CHELSEA234 For the data augmentation, following the paper, I used flipping, rotation (90 180 270 degree) and downsizing (scaling factor from 1 to 0.5).

For the matlab.engine, I did that in another testing script by converting one jpg image into numpy array using matlab in python using codes similar as:

im_gt, im_b, im_gt_y, im_b_y = eng.rgb_array(image_name, matlab.double([[scale]]), nargout=4)
im_gt_y = np.array(im_gt_y._data).reshape(im_gt_y.size, order='F')
im_b_y = np.array(im_b_y._data).reshape(im_b_y.size, order='F')

[yuanshuai220]

I trained the VDSR for 50 epochs, and the PSNR was 37.47dB on Set5. But when I use the model you provided, the PSNR was 36.65dB. I think maybe the batchSize or learning rate influences the result, could you please tell your implement settings.

[twtygqyy]

@CHELSEA234 All hyperparameters I used for training are provided in this repository with default settings. I guess the only reason leads to the difference is because the way of calculating PSNR and matlab bicubic interpolations

[yuanshuai220]

@twtygqyy Thanks for reply. I just find my batchsize is 64 and the default is 128. I will use the default settings train again.

[Overtown]

@yuanshuai220 Could you tell me how about the result when you used 128 batchsize?

[wenyuqing]

请问您这里是不是没有生成.mat文件的代码啊