Appraise

Appraise copied to clipboard

Can't create new evaluation tasks

1

Hello, I can't access the page to add our own evaluation task. I'm currently using Python 2.7 and followed the solution found at https://github.com/cfedermann/Appraise/issues/51#issuecomment-712405594. But once I'm in the adming...

Nolanogenn

Incorrect Django requirements on readme

6

Readme states Appraise should work with Django >1.3, but it does not work with any 1.3 version, because certain symbols imported on urls.py were on _django.conf.urls.default_ on 1.3, but on...

dtr5

Minor error in settings.py prevents collectstatic to work

1

In settings.py, STATIC_ROOT = os.path.join(ROOT_PATH, '/static-files'), remove the leading slash, since otherwise os.path.join just returns /static-files.

jmcoursi

Hi, I wonder if we can create our own evaluation tasks with Appraise? Following the ReadMe, I will only have a example homepage for WMT2016, but I will not be...

zmykevin

We need to be able to create actual bad refs, similar to what Yvette's code did for WMT17. Might be easy to simply integrate her scripts?

cfedermann

We need a better way to view progress for campaigns. Based on required annotations per system, it should be fairly straightforward to render a convergence graph for a given campaign.

cfedermann

It should be possible from the Django admin backend to retire a campaign. This should also retire any associated objects such as tasks, items, and maybe results. The corresponding campaign...

cfedermann

If input data is exactly identical (= same JSON, same IDs, same campaign) then we should not create redundant task instances. This only pollutes the database.

cfedermann

How to compute the system ranking clusters if systems often produce the same output and are merged in the results CSV file? Is using the [`scripts/compute_ranking_clusters.perl`](https://github.com/cfedermann/Appraise/blob/master/scripts/compute_ranking_clusters.perl) script the correct way?...

tuetschek

When installing and running the `collectstatic` command, it couldn't find the `jquery.js`. Do I have to install other packages other than `django==1.3`? ``` ~/Appraise-Software/appraise$ python manage.py collectstatic WARNING:appraise.utils:NLTK is NOT...

alvations