BAPCtools
BAPCtools copied to clipboard
Clean up timeouts
- There is the global program timeout for generators/visualizers/building (i.e.anything not submissions), which defaults to
30sand is overridden by--timeoutfor e.g.bt generate - There is a per problem submission timeout, which is derived from the timelimit and overridden using
--timeoutonbt run.
This makes for some inconsistency because now timeout is both a program and problem property.
We could 'fix' the code and always make timeout a program property, which is either derived from the global timeout or from the problem timeout, but I'm not sure that's actually worth fixing.
I think we should fix the duplicate --timeout flag name, since this makes bapctools.yaml configuration a bit annoying otherwise. I'm not exactly sure how though. One option is to give a different name to one of them, but have -T as shortcut for both anyway. (-t is currently the shortcut for timelimit already.)
I'm not sure whether --timelimit is more common/useful for the problem or generate case. For problems, I typically just change the timelimit, not the timeout, if I want to see how long things take. On the other hand, although rare, I could see generators taking more than 30 seconds.
CC @mpsijm
For problems, I typically just change the timelimit, not the timeout, if I want to see how long things take.
I agree with that one, so in that sense I'd say we could rename/rework how the timeout works. As I have written in https://github.com/RagnarGrootKoerkamp/BAPCtools/pull/163#discussion_r748593805, we could change the submission timeout to work more like Kattis' safety_margin, so that the distinction with a Program timeout is clearer?
Implementing the limits: options as problemtools uses them is a long outstanding issue anyway, so safety_margin is definitely something we should do. Still, on the command line you would want to pass an absolute time, not a factor, I think.
Thinking about it, timeout: in a bapctools.yaml really should be for the generators, not the problem. So maybe the cleanest is to just drop the run --timeout flag (or make it do the same as generate --timeout), and force the user to do this using safety_margin in the problem config or by just doing -t/--timelimit. Alternatively, we could make --timeout override both generator and submission timeouts when it is specified on the command line, but that's probably confusing.