gec-ranking icon indicating copy to clipboard operation
gec-ranking copied to clipboard

Update Request: correcting the formula in GLEU+ paper

Open shotakoyama opened this issue 1 year ago • 0 comments

The GLEU+ paper (https://arxiv.org/pdf/1605.02592.pdf) shows that the formula of precision $p_k$ is:

$$ p_n = \cfrac{ \displaystyle \sum_i^{|D|} \sum_{g_n \in h_i} \min(\rho_{i,g_n}, \eta_{i, g_n}) - \max(0, \min(\sigma_{i, g_n}, \eta_{i, g_n}) - \min(\rho_{i, g_n}, \eta_{i, g_n}))} {\displaystyle \sum_i^{|D|} \sum_{g_n \in h_i} \eta_{i, g_n} } $$

However, GLEU+ in your source code is:

$$ p_n = \cfrac{ \displaystyle \sum_i^{|D|} \max(0, \sum_{g_n \in h_i} \min(\rho_{i,g_n}, \eta_{i, g_n}) - \min(\sigma_{i, g_n}^{\mathrm{diff}}, \eta_{i, g_n})) } {\displaystyle \sum_i^{|D|} \sum_{g_n \in h_i} \eta_{i, g_n} } $$

(notation: $\sigma_{i,g_n}=\mathrm{count}(g_n \in s_i)$, $\rho_{i,g_n}=\mathrm{count}(g_n \in r_i)$, $\eta_{i,g_n}=\mathrm{count}(g_n \in h_i)$, $\sigma_{i,g_n}^{\mathrm{diff}}=\sigma_{i,g_n}$ if $\rho_{i,g_n}=0$ else $0$)

Could you provide an update on the formula explanation?

shotakoyama avatar Jun 11 '23 23:06 shotakoyama