gerbil
gerbil copied to clipboard
dice-group
experiment with errors
Hi,
I submitted a job in gerbil with 3 questions to test and after a couple of times getting errors such as "The annotator caused too many single errors", now the job seems to be running indefinitely.
You can follow it here: http://gerbil-qa.aksw.org/gerbil/experiment?id=201902210063
If it finishes, it will probably show that error I mentioned. Can someone tell me why is it still running and point out what are exactly the errors about?
I uploaded my own dataset, but even if I use one of the available datasets, as the QALD6 Test Multilingual, the same thing happens.
Thank you.
Hi, I will check that out. Just in case: Can you provide me the file? (3 questions is easier to debug than a complete QALD 6 file) Otherwise i will just check with the QALD 6 dataset.
YODA does not work since 1-2 years or so. they shut down their demo which was working with gerbil and replaced it with some strange one. SINA had always problems, HAWK and QANARA/wdaqua changed url/ip OKBQA changes url also very often. (i did not check, but it seems to be the problem) ganswer2 seems to be down from the debug messages.
all new IPs are not in the online release yet.
and if i saw correctly: the max amount of errors is set to 1. @MichaelRoeder is that correct? it seems off to me.
There is a maximum amount of questions that are allowed to fail. This is currently hard coded to 0.25. If 1of 3 questions causes an error (i.e., an exception) this threshold is already reached.
OK. So we just need to change the URLs to the correct ones. We have to think of a better way than change the URLs every half a year :/
Ok. That means that my sample is correct, right? Did you guys update the URLs already? I was testing again with same errors. Thank you very much.
I updated every URL/IP i could find, and pushed it, did not update the online version yet though. Will do that today then close the issue.
Dear all,
Is it true that only QUEPY system is working for QA in gerbil after the URLs update? I uploaded my dataset and could get some results with 24 errors only by removing the dates in my queries with QUEPY system. Can you tell me what are theses errors about so that I can adapt my data accordingly if needed?
You see the results for this in the URL: http://gerbil-qa.aksw.org/gerbil/experiment?id=201904080006
Best regards!
Tarcísio Souza PhD Candidate L3S Research Center / Leibniz Uni. 15th Floor, Room: 1519 Appelstraße 9 30167 Hannover, Germany Am 6. Apr. 2019, 14:39 +0200 schrieb F. Conrads [email protected]:
Closed #311. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.
Hi, indeed these numbers do not look ok. @LukasBluebaum can you have a look @TortugaAttack maybe you can upload the log information here? Do not know whether this is useful or not.
