Biased gradients

[manuel-delverme]

you need to re-evaluate the features/state after the optimization step optim.step() because that updates the feature layer hence the features themselves

[lweitkamp]

Hey @manuel-delverme. Do you mean moving lines 112-114

state = option_critic.get_state(to_tensor(next_obs))
option_termination, greedy_option = option_critic.predict_option_termination(state, current_option)

following the optimisation step?

[lweitkamp]

Fixed as of the latest commit