[Question] Multiple Callbacks For callback_on_new_best

[EitanGronich]

❓ Question

I have a simple use case that I have found no answer to in the documentation. (v. 2.2.1) I want to pass two chained callbacks as a callback_on_new_best for EvalCallback. (A custom one for drawing a figure and a library one - StopTrainingOnRewardThreshold) Passing a CallbackList doesn't seem to work, since the parent attribute of the callbacks is None, and the code throws an AssertionError because it asserts their parent is the EvalCallback. There seem to be no other options except building a custom callback that does them both, but this doesn't seem like good practice given that CallbackList does exist. Am I missing something? Minimal reproducible:

import gymnasium as gym
from stable_baselines3 import PPO
from stable_baselines3.common.callbacks import BaseCallback, EvalCallback, StopTrainingOnRewardThreshold, CallbackList

class CustomCallback(BaseCallback):

    def __init__(self, verbose=0):
        super().__init__(verbose)
    
    def _on_step(self):
        # Draw a figure
        return True


eval_env = gym.make("Pendulum-v1")

stop_on_threshold_callback = StopTrainingOnRewardThreshold(reward_threshold=-200, verbose=1)

eval_callback = EvalCallback(eval_env, best_model_save_path="./logs/",
                             log_path="./logs/", eval_freq=500,
                             deterministic=True, render=False,
                             callback_on_new_best=CallbackList([CustomCallback(), stop_on_threshold_callback])
                             )

model = PPO("MlpPolicy", "Pendulum-v1")
model.learn(5000, callback=eval_callback)

Thanks a lot!

Checklist

• [X] I have checked that there is no similar issue in the repo
• [X] I have read the documentation
• [X] If code there is, it is minimal and working
• [X] If code there is, it is formatted using the markdown code blocks for both code and stack traces.

[araffin]

Hello, thanks for reporting the issue, I think we need to give access to the common parent when using a CallbackList.

[icheered]

Running into the same issue! Is there really no work around?

[araffin]

Is there really no work around?

a quick workaround is to redefine StopTrainingOnRewardThreshold, combine the two callbacks in one

a better workaround is to fix the code and I would appreciate to receive a PR that solves this issue ;)

[will-maclean]

I'm happy to work on implementing this one