eval-dev-quality icon indicating copy to clipboard operation
eval-dev-quality copied to clipboard

Do not copy assessments of base tasks

Open ahumenberger opened this issue 9 months ago • 1 comments

Please help us keep the number of duplicated issues small.

Brief Description

The write-tests-symflower-template task is based on the write-tests task. As such it inherits all the assessments of the base task.

If a certain language does not support templates, then it still inherits the assessments, which leads to wrong assessments.

Reproducer

No response

Logs


Additional Information

No response

Version

1.0

ahumenberger avatar Mar 21 '25 07:03 ahumenberger

There are two problems here:

  • The assessments should not be copied. They are their own task completely. Own queries execution and everything. they only share repository and case, and maybe some other data.
  • If a language doesn't support a feature for a task, it should not be executed at all

zimmski avatar Apr 08 '25 15:04 zimmski