Tool for auto-tuning time limits

[edomora97]

This is the tracking issue for a tool that selects the best time limit to use for a task, such that every solution gets the correct subtasks.

Inputs

• Solutions with the set of subtask on which they should get AC, and on which they should get TLE (possible thanks to #34)
• A maximum possible time limit
• A specification for computing the bounds

Outputs

• An interval in which the solutions get the correct subtasks

---

After running the solutions with the maximum possible time limit, we know the run times of each solution, of each subtask. Solutions that take more than this time limit can be considered to take this time.

For sure, the resulting time limit should be greater or equal to the maximum time of all the solutions that should get AC. Similarly, it should be lower than the maximum time of each subtask that should get TLE.

This can either produce an interval (a solution exists), or no interval (no solution exists). However, it is beneficial to restrict the bounds further. For example, you may want to add the bound that the result is greater or equal to "twice" the maximum time of all the solutions that should get AC (to be more confident that a solution doesn't get spurious TLE). Similarly, you may want to restrict the upper bound in similar way.

---

Open questions

1. How the "strict" bounds should be specified?
   • Simply a factor? (e.g. 2x in the previous example)
   • A factor and a minimum bound? (e.g. 2x the time, at least +0.5s)
2. And the upper bound?
   • Still a factor?
   • A constant? (e.g. at least +0.5s)
3. This tool should produce just an interval or maybe also a proposed time?
   • Maybe it should produce both a "loose" and a "strict" interval
   • Which are the rules for rounding the proposed time?
4. Name of the tool?
   • task-maker-tools auto-tune-time-limit
   • task-maker-tools auto-tune
   • task-maker-tools find-time-limit
   • task-maker-tools 🐟
   • Other proposals?