RandBox icon indicating copy to clipboard operation
RandBox copied to clipboard

Questions about re-implementation

Open insanebai opened this issue 10 months ago • 1 comments

Hello author, I conducted experiments according to your file configuration (completely consistent settings) and found that the experimental results are different from the paper results.

Task t1: AOSE is about 1500 higher K-map is 4 points lower UR is 3 points lower What causes this?

insanebai avatar Apr 29 '24 05:04 insanebai

[05/06 10:32:54 d2.evaluation.evaluator]: Total inference time: 1:46:22.203930 (0.704982 s / iter per device, on 1 devices) [05/06 10:32:54 d2.evaluation.evaluator]: Total inference pure compute time: 1:45:54 (0.701939 s / iter per device, on 1 devices) Wilderness Impact: {0.1: {50: 0.019775390624999997}, 0.2: {50: 0.026684253246753248}, 0.3: {50: 0.03153739177246422}, 0.4: {50: 0.029262144473296583}, 0.5: {50: 0.025682171277689847}, 0.6: {50: 0.024073368590892794}, 0.7: {50: 0.01802047416673349}, 0.8: {50: 0.01965570856271746}, 0.9: {50: 0.02210940767555562}} avg_precision: {0.1: {50: 0.09990769012284315}, 0.2: {50: 0.09990769012284315}, 0.3: {50: 0.09990769012284315}, 0.4: {50: 0.09990769012284315}, 0.5: {50: 0.09990769012284315}, 0.6: {50: 0.09990769012284315}, 0.7: {50: 0.09990769012284315}, 0.8: {50: 0.09990769012284315}, 0.9: {50: 0.09990769012284315}} Absolute OSE (total_num_unk_det_as_known): {50: 9528.0} total_num_unk 23320 AP50: ['67.6', '47.1', '49.4', '29.2', '21.3', '60.4', '48.1', '78.2', '15.7', '50.8', '17.5', '66.8', '66.8', '56.2', '45.5', '21.1', '55.5', '38.5', '71.9', '50.4', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.9'] Precisions50: ['6.6', '6.1', '5.2', '2.5', '2.3', '11.4', '6.1', '16.3', '2.7', '3.7', '5.5', '11.5', '8.6', '8.5', '6.5', '3.5', '4.0', '6.8', '16.2', '8.8', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '10.0'] Recall50: ['92.8', '70.7', '75.0', '68.6', '44.3', '79.4', '67.8', '91.4', '45.6', '89.3', '38.2', '90.7', '90.2', '80.5', '67.4', '67.5', '86.5', '71.6', '86.0', '73.7', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '0.0', '6.0'] Current class AP50: 47.89914697917603 Current class Precisions50: 7.141895371900378 Current class Recall50: 73.84517149369682 Known AP50: 47.89914697917603 Known Precisions50: 7.141895371900378 Known Recall50: 73.84517149369682 Unknown AP50: 0.8798663376498874 Unknown Precisions50: 9.980846988721003 Unknown Recall50: 6.0334476843910805

我的结果是这样的,哥哥

WangPingA avatar May 06 '24 04:05 WangPingA