REDN
REDN copied to clipboard
Is the score evaluation function correct?
gold_count = len(gold_label)
for idx, s in enumerate(res):
if idx in gold_label and s == 1:
self.res[CORRECT] += 1
self.triple_count_res[triple_count][CORRECT] += 1 if idx != self.na_id else 0
self.without_na_res[CORRECT] += 1 if idx != self.na_id else 0
self.na_res[CORRECT] += 1 if idx == self.na_id else 0
self.normal_res[CORRECT] += 1 if is_normal_data and idx != self.na_id else 0
self.multi_label_res[CORRECT] += 1 if is_multi_label_data and idx != self.na_id else 0
self.over_lapping_res[CORRECT] += 1 if is_over_lapping_data and idx != self.na_id else 0
gold_count -= 1
if idx in gold_label:
self.res[GOLD_POSITIVE] += 1
self.triple_count_res[triple_count][GOLD_POSITIVE] += 1 if idx != self.na_id else 0
self.without_na_res[GOLD_POSITIVE] += 1 if idx != self.na_id else 0
self.na_res[GOLD_POSITIVE] += 1 if idx == self.na_id else 0
self.normal_res[GOLD_POSITIVE] += 1 if is_normal_data and idx != self.na_id else 0
self.multi_label_res[GOLD_POSITIVE] += 1 if is_multi_label_data and idx != self.na_id else 0
self.over_lapping_res[GOLD_POSITIVE] += 1 if is_over_lapping_data and idx != self.na_id else 0
if s == 1:
self.res[PRED_POSITIVE] += 1
self.triple_count_res[triple_count][PRED_POSITIVE] += 1 if idx != self.na_id else 0
self.without_na_res[PRED_POSITIVE] += 1 if idx != self.na_id else 0
self.na_res[PRED_POSITIVE] += 1 if idx == self.na_id else 0
self.normal_res[PRED_POSITIVE] += 1 if is_normal_data and idx != self.na_id else 0
self.multi_label_res[PRED_POSITIVE] += 1 if is_multi_label_data and idx != self.na_id else 0
self.over_lapping_res[PRED_POSITIVE] += 1 if is_over_lapping_data and idx != self.na_id else 0
self.res[TOTAL] += 1 # shouldn't here to add "gold_count" instead of 1?
self.triple_count_res[triple_count][TOTAL] += 1 if ep[2] != self.na_id else 0
self.without_na_res[TOTAL] += 1 if ep[2] != self.na_id else 0
self.na_res[TOTAL] += 1 if ep[2] == self.na_id else 0
self.normal_res[TOTAL] += 1 if is_normal_data and ep[2] != self.na_id else 0
self.multi_label_res[TOTAL] += 1 if is_multi_label_data and ep[2] != self.na_id else 0
self.over_lapping_res[TOTAL] += 1 if is_over_lapping_data and ep[2] != self.na_id else 0
In your f1_metric.py, when calculating the score, as self.res[TOTAL] += 1 instead of gold_count, it is possible that your accuracy exceeded 100% (if gold_count >= 2). I wonder if my understanding is wrong?