LIVE dataset test result

[mlkk518]

Hello! Sorry to bother you! I run your code without any changes except for increasing the epoch from 10 to 20 and reducing the number of repetition from 10 to 2. However, what makes me upset is that I could hardly get the same metric values which just near yours ,such as PLCC and SROCC. Both LIVE dataset and ResNet network are choosed for the experienment. Here is the results: `( Parameters: LIVE; ResNet; bins = 5; patches = 50; SRCC: 0.9478; 0.9494; 0.9514; 0.9522; 0.9527; 0.9509; 0.9510; 0.9509; 0.9513; 0.9521; std_SRCC: 0.0031; 0.0005; 0.0006; 0.0030; 0.0012; 0.0012; 0.0003; 0.0005; 0.0015; 0.0007; PLCC: 0.9556; 0.9570; 0.9587; 0.9589; 0.9602; 0.9582; 0.9589; 0.9581; 0.9589; 0.9593; std_PLCC: 0.0057; 0.0032; 0.0051; 0.0073; 0.0047; 0.0054; 0.0046; 0.0047; 0.0058; 0.0051; best_SRCC: 0.9527; best_PLCC: 0.9602

Parameters: LIVE; ResNet; bins = 5; patches = 50; SRCC: 0.9457; 0.9457; 0.9503; 0.9504; 0.9522; 0.9514; 0.9510; 0.9510; 0.9517; 0.9527; 0.9523; 0.9530; 0.9526; 0.9518; 0.9512; 0.9525; 0.9515; 0.9518; 0.9520; 0.9524; std_SRCC: 0.0004; 0.0045; 0.0004; 0.0004; 0.0007; 0.0022; 0.0013; 0.0006; 0.0008; 0.0009; 0.0000; 0.0011; 0.0002; 0.0008; 0.0010; 0.0011; 0.0000; 0.0003; 0.0005; 0.0022; PLCC: 0.9547; 0.9551; 0.9572; 0.9575; 0.9593; 0.9579; 0.9581; 0.9577; 0.9579; 0.9587; 0.9589; 0.9590; 0.9594; 0.9584; 0.9586; 0.9590; 0.9580; 0.9584; 0.9583; 0.9587; std_PLCC: 0.0032; 0.0001; 0.0036; 0.0042; 0.0025; 0.0050; 0.0029; 0.0031; 0.0045; 0.0032; 0.0039; 0.0029; 0.0028; 0.0033; 0.0020; 0.0023; 0.0035; 0.0031; 0.0033; 0.0050; best_SRCC: 0.9530; best_PLCC: 0.9590 `)

 Could you please tell me what's wrong about my setting .I am looking forward for your reply, thanks a lot . 

[HuiZeng]

Hi, I has been several years that I did not work on the IQA topic. If I remember correctly, the variance is very large when using different random splits. That may be the main reason. You can use more repetitions to check it.

[mlkk518]

Thanks a lot  for your reply quickly , I would try the next. 

------------------ 原始邮件 ------------------ 发件人: "Hui Zeng"<[email protected]>; 发送时间: 2020年6月4日(星期四) 下午3:45 收件人: "HuiZeng/BIQA_Toolbox"<[email protected]>; 抄送: "1198484350"<[email protected]>;"Author"<[email protected]>; 主题: Re: [HuiZeng/BIQA_Toolbox] LIVE dataset test result (#6)

Hi, I has been several years that I did not work on the IQA topic. If I remember correctly, the variance is very large when using different random splits. That may be the main reason. You can use more repetitions to check it.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.