eval-dev-quality icon indicating copy to clipboard operation
eval-dev-quality copied to clipboard

Extend the evalatuion.csv with the current maximum values, and keep them updated in the repositories

Open zimmski opened this issue 11 months ago • 0 comments

Please help us keep the number of duplicated issues small. We kindly ask you to add your input to the appropriate issue or PR in case your feature idea is already being tracked.

Problem

Maximum metrics can change over time for a language, repository, task, case, ... to freeze the maximums, we want to save them in the evaluation.csv

Solution (optional)

Tasks:

  • [ ] Read maximums from the repository files
  • [ ] Insert
    • [ ] add files-executed-maximum
    • [ ] add coverage-maximum
    • [ ] add tests-passing-maximum
    • [ ] add response-no-error-maximum
    • [ ] add response-with-code-maximum
    • [ ] add response-no-excess-maximum
  • [ ] Update maximums if the value gets bigger in the repository configuration

zimmski avatar Jan 30 '25 17:01 zimmski