Alexander Klenin
Alexander Klenin
Perhaps unify with `columns` in cats-judge
Similar to limit overrides.
For a specific judge. Is useful in case of broken cache state, changes in judge config and as a partial solution to #203.
This is not easy, since judges are not guaranteed no be online in the moment of deletion, and judge can receive same problem from multiple sources, including local files.
Currently, some testset errors only result in a server logs warnings. (E.g. recursive scoring group, unparseable testset spec in request override). They should be displayed in the UI.
Currently it is possible to either enter scoring group name, invoking scoregroup plan, or enter test numbers, causing efficiantly 'all' plan. It could be useful to include other combination, in...