Appraise
Appraise copied to clipboard
cfedermann
Appraise evaluation system for manual evaluation of machine translation output
I don't see which group I am part of. (Silly workaround: do 1 HIT and check which group's progress has increased.) The name of my group could be shown: -...
After downloading and running Appraise it seems that the current code is intended to work only for WMT15 evaluation. I would like to create my own evaluation tasks but [the...
Hello, Can you please elaborate what license is Appraise under? I didn't see any license information or file in the repo? Thanks
We have seen spaces, non-ASCII characters and symbols resulting in server errors during signup POST submission. Fix this and give better error message.
Many times, the systems outputs for a sentence are identical. Rather than constructing each task from a random subset of _systems_, each task should be constructed from the set of...
Compute inter-annotator agreement (IAA) for the number of annotators _or coders_ (C) which maximises the number of items (I) that have been evaluated by the respective sub set of coders....
Instead of having various file level scripts for shell admin ops, add custom management commands and clean up things... See Django documentation here: - https://docs.djangoproject.com/en/dev/howto/custom-management-commands/
Use random sample of 10 HITs per WMT14 language pair and allow infinite collection of annotation results on these... Create DEMO group for demo users.
The web application crashes with Django 1.5. This is the error shown in the browser: NoReverseMatch at /appraise/ 'url' requires a non-empty first argument. The syntax changed in Django 1.5,...
Appraise requires Django >=1.3 but the web app doesn't work with 1.3.1. When pointing the browser to "http://127.0.0.1:8000/appraise/" with the server running, I get the following error: ImportError at /appraise/...
Metadata
Owner
Metadata
Appraise evaluation system for manual evaluation of machine translation output