BAPCtools
BAPCtools copied to clipboard
Clean up timeouts
- There is the global program timeout for generators/visualizers/building (i.e.anything not submissions), which defaults to
30s
and is overridden by--timeout
for e.g.bt generate
- There is a per problem submission timeout, which is derived from the timelimit and overridden using
--timeout
onbt run
.
This makes for some inconsistency because now timeout is both a program and problem property.
We could 'fix' the code and always make timeout a program property, which is either derived from the global timeout or from the problem timeout, but I'm not sure that's actually worth fixing.
I think we should fix the duplicate --timeout
flag name, since this makes bapctools.yaml
configuration a bit annoying otherwise. I'm not exactly sure how though. One option is to give a different name to one of them, but have -T
as shortcut for both anyway. (-t
is currently the shortcut for timelimit already.)
I'm not sure whether --timelimit
is more common/useful for the problem or generate case. For problems, I typically just change the timelimit, not the timeout, if I want to see how long things take. On the other hand, although rare, I could see generators taking more than 30 seconds.
CC @mpsijm
For problems, I typically just change the timelimit, not the timeout, if I want to see how long things take.
I agree with that one, so in that sense I'd say we could rename/rework how the timeout works. As I have written in https://github.com/RagnarGrootKoerkamp/BAPCtools/pull/163#discussion_r748593805, we could change the submission timeout to work more like Kattis' safety_margin
, so that the distinction with a Program timeout is clearer?
Implementing the limits:
options as problemtools uses them is a long outstanding issue anyway, so safety_margin
is definitely something we should do. Still, on the command line you would want to pass an absolute time, not a factor, I think.
Thinking about it, timeout:
in a bapctools.yaml
really should be for the generators, not the problem. So maybe the cleanest is to just drop the run --timeout
flag (or make it do the same as generate --timeout
), and force the user to do this using safety_margin
in the problem config or by just doing -t/--timelimit
. Alternatively, we could make --timeout
override both generator and submission timeouts when it is specified on the command line, but that's probably confusing.