PaddleNLP icon indicating copy to clipboard operation
PaddleNLP copied to clipboard

[UIE Issue]UIE关系抽取后执行evaluate.py文件发现,对性能评估的问题:关系的类居然是一条一条class 输出的,不应该是都属于一类输出吗

Open tingaicompass opened this issue 3 years ago • 3 comments

记得很早之前看过一个关于UIE关系抽取后执行evaluate.py文件发现,对性能评估的问题。

以这篇项目为例: Paddlenlp之UIE关系抽取模型【高管关系抽取为例】 https://aistudio.baidu.com/aistudio/projectdetail/4371345?contributionType=1 项目中结果验证时: Class Name: 人名 [2022-07-25 00:17:26,608] [ INFO] - Evaluation Precision: 1.00000 | Recall: 1.00000 | F1: 1.00000 [2022-07-25 00:17:26,641] [ INFO] - ----------------------------- [2022-07-25 00:17:26,641] [ INFO] - Class Name: 公司 [2022-07-25 00:17:26,641] [ INFO] - Evaluation Precision: 1.00000 | Recall: 0.66667 | F1: 0.80000 [2022-07-25 00:17:26,667] [ INFO] - ----------------------------- [2022-07-25 00:17:26,667] [ INFO] - Class Name: 网易公司的高管 [2022-07-25 00:17:26,667] [ INFO] - Evaluation Precision: 1.00000 | Recall: 1.00000 | F1: 1.00000 [2022-07-25 00:17:26,693] [ INFO] - ----------------------------- [2022-07-25 00:17:26,693] [ INFO] - Class Name: 百度公司的高管 [2022-07-25 00:17:26,693] [ INFO] - Evaluation Precision: 1.00000 | Recall: 1.00000 | F1: 1.00000

关系的类居然是一条一条class 输出的,不应该是都属于一类输出吗 百度公司的高管 网易公司的高管 应该输出为:Class Name:高管 Evaluation Precision: 1.00000 | Recall: 1.00000 | F1: 1.00000

不知道更新优化了吗

tingaicompass avatar Sep 08 '22 06:09 tingaicompass

evaluate.py debug输出的各类别集合和数据集内"prompt"字段内容集合是一样的

MachineSheep avatar Sep 08 '22 06:09 MachineSheep

evaluate.py debug输出的各类别集合和数据集内"prompt"字段内容集合是一样的

那样不是很不友好呀,关系抽取得prompt好像是 “” xxx的xxx“” 做关系抽取关系数据,希望看到所有“”高管“”关系抽取的ACC等指标,这样一条一条看有点麻烦呀:

tingaicompass avatar Sep 08 '22 06:09 tingaicompass

感谢反馈~我们优化下这部分体验 @dingidng

linjieccc avatar Sep 08 '22 07:09 linjieccc

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动,被标记为stale。

github-actions[bot] avatar Dec 07 '22 07:12 github-actions[bot]

This issue was closed because it has been inactive for 14 days since being marked as stale. 当前issue 被标记为stale已有14天,即将关闭。

github-actions[bot] avatar Dec 21 '22 11:12 github-actions[bot]