rlberry icon indicating copy to clipboard operation
rlberry copied to clipboard

Stopping criterion utility

Open TimotheeMathieu opened this issue 2 years ago • 3 comments

Implement a tool to stop the algorithm when some value (in the writer) go above some threshold. Typically for either early stopping or for stopping after a certain number of episodes.

TimotheeMathieu avatar Jul 10 '23 09:07 TimotheeMathieu

Should we modify the signature of the fit method to be somth likedef fit(self, budget=100, stop_callback: Callable[[Writer], bool]) ?

riiswa avatar Jul 10 '23 13:07 riiswa

I think there is not a simple way to solve this PR. Each agent have its own implementation of the fit method, so the early stopping criterion should be handled in each agents...

riiswa avatar Jul 24 '23 11:07 riiswa

My idea was the following:

  • Have an optional argument in the fit that default to None, as you said it can be something like stop_callback: None or Callable[[Writer], bool]. This is optional, users can implement an agent without using it, the same way as users can use the budget or not (see ValueIteration for instance).
  • Have a prototype of callback function that can be used for anyone and is well documented.
  • Have the function implemented in all rlberry agents to show how it is done.

This should be sufficient, we don't need to have something automatic, we only need to make it simple to use and have it implmented in the rlberry agents.

TimotheeMathieu avatar Jul 24 '23 12:07 TimotheeMathieu