wl-coref
wl-coref copied to clipboard
vdobrovolskii
Evaluation problem
It seems like doing training is fine, but while doing evaluation, for some of documents it works but it does not work and generate the following error:
====> (nw/wsj/00/wsj_0049); part 000: File (nw/wsj/00/wsj_0049); part 000: Entity 0: (0,3) (73,73) (221,222) (237,238) (281,282) (326,326) (402,403) (456,456) (524,525) (578,578) (605,605) (636,637) (645,645) (653,654) (699,701) (770,771) Entity 1: (5,5) (28,47) (72,74) Entity 2: (14,26) (62,63) (619,621) (714,716) (718,719) (757,758) Entity 3: (44,46) (704,706) Entity 4: (67,70) (228,231) Entity 5: (77,121) (191,192) Entity 6: (81,81) (455,458) Entity 7: (92,95) (130,132) Entity 8: (113,114) (148,149) Entity 9: (116,121) (123,124) (154,155) (247,248) Entity 10: (151,155) (170,171) Entity 11: (167,167) (874,874) (934,934) Entity 12: (184,184) (197,197) Entity 13: (188,194) (453,453) (648,648) Entity 14: (218,219) (242,244) Entity 15: (250,251) (261,261) Entity 16: (266,278) (299,331) (333,334) (367,368) (412,412) Entity 17: (272,278) (415,419) Entity 18: (294,295) (382,383) (1738,1739) Entity 19: (343,352) (354,355) (363,363) Entity 20: (378,378) (386,386) (398,399) (409,410) Entity 21: (429,430) (464,465) (1270,1271) (1444,1445) (1918,1919) Entity 22: (438,439) (542,543) (588,589) (778,779) Entity 23: (441,445) (469,471) (482,484) Entity 24: (460,475) (516,517) (519,519) (551,551) (558,558) (575,576) Entity 25: (489,489) (499,499) Entity 26: (492,493) (503,504) Entity 27: (520,520) (548,549) Entity 28: (527,539) (554,556) (581,584) (685,688) (692,692) Entity 29: (593,598) (661,665) (674,675) (774,775) Entity 30: (650,654) (669,669) Entity 31: (708,708) (712,712) Entity 32: (730,730) (747,747) Entity 33: (815,816) (833,841) (846,847) (854,854) (886,886) (893,893) (901,901) (904,904) (917,918) Entity 34: (862,866) (1026,1030) Entity 35: (883,888) (911,911) Entity 36: (924,925) (942,942) Entity 37: (948,952) (1010,1010) (1032,1032) (1039,1040) (1044,1044) (1053,1053) Entity 38: (958,958) (962,962) Entity 39: (966,966) (971,971) Entity 40: (1065,1066) (1074,1076) (1101,1101) (1123,1124) (1129,1136) (1173,1173) (1176,1177) (1186,1187) (1229,1230) (1232,1234) (1320,1321) Entity 41: (1065,1068) (1110,1111) Entity 42: (1079,1084) (1091,1091) (1133,1134) (1157,1158) (1200,1201) (1203,1205) (1247,1249) (1294,1295) (1313,1314) Entity 43: (1086,1088) (1303,1304) Entity 44: (1097,1097) (1242,1243) Entity 45: (1113,1127) (1144,1155) (1183,1183) (1194,1195) Entity 46: (1160,1167) (1179,1180) Entity 47: (1173,1174) (1209,1210) (1306,1307) Entity 48: (1197,1201) (1212,1212) Entity 49: (1232,1240) (1258,1259) (1273,1283) Entity 50: (1255,1256) (1291,1295) Entity 51: (1323,1324) (1332,1356) Entity 52: (1325,1325) (1331,1331) Entity 53: (1327,1330) (1436,1445) (1447,1448) Entity 54: (1344,1345) (1373,1378) (1414,1416) Entity 55: (1358,1368) (1397,1398) Entity 56: (1480,1480) (1502,1504) Entity 57: (1482,1482) (1608,1610) Entity 58: (1482,1483) (1493,1497) (1506,1506) (1541,1542) (1593,1594) (1605,1605) (1661,1663) Entity 59: (1485,1485) (1630,1630) Entity 60: (1489,1491) (1533,1535) (1548,1549) (1556,1556) (1564,1564) Entity 61: (1506,1510) (1661,1664) Entity 62: (1578,1591) (1635,1636) (1657,1658) Entity 63: (1623,1624) (1639,1639) Entity 64: (1697,1697) (1713,1714) (1742,1752) Entity 65: (1701,1702) (1706,1714) (1801,1802) Entity 66: (1721,1721) (1923,1923) Entity 67: (1742,1759) (1774,1775) (1792,1792) Entity 68: (1764,1770) (1783,1784) Entity 69: (1829,1837) (1873,1877) Entity 70: (1839,1861) (1879,1879) (1890,1894) ====> (nw/wsj/00/wsj_0049); part 000: Detected the end of a mention 30 without begin (?,1136) at reference-coreference-scorers/lib/CorScorer.pm line 292, <F> line 78226.
Do you know what causes that?
Hi! I'm not sure why this happens. Could you please share everything I might need to reproduce the error?
Hi,
The files were too large, so I can also share the google drive https://drive.google.com/drive/folders/1l0xC-_WYRu7C-FYnC2k8BWGA2SdBkBoR?usp=sharing with you.
Best, Andy Liu
liu Andy @.***> 于2023年2月10日周五 04:25写道：
Hi,
I was trying using Roberta-large proposed in your paper, and I also tried deberta-large-v3. For some epochs, the evaluation is unsuccessful. I am using the same training code. I have also attached the conll.py which I looked at the previous responses and issues. I did not modify any other parts of the code. For epoch 12, the evaluation is successful while for epochs 13 -20, the evaluation is unsuccessful. I am not sure what might cause this problem. I would really appreciate it if you are willing to have a look at the attached files. And if you would like to see any others, please let me know.
Best, Andy Liu
vdobrovolskii @.***> 于2023年2月3日周五 09:28写道：
Hi! I'm not sure why this happens. Could you please share everything I might need to reproduce the error?
— Reply to this email directly, view it on GitHub https://github.com/vdobrovolskii/wl-coref/issues/40#issuecomment-1415946237, or unsubscribe https://github.com/notifications/unsubscribe-auth/AT3IT6LFBGDX6ZKWOZU2YFTWVUIZHANCNFSM6AAAAAAUF3E7GI . You are receiving this because you authored the thread.Message ID: @.***>
Hi I'm sorry I disappeared and lost the track of time. Could you kindly confirm seeking the solution to the problem is still relevant to you?