agents
agents copied to clipboard
Switch tf.losses.mse with tf.math.squared_difference in BehavioralCloningAgent
When training a BC Agent with a continuous action space, I received the following warning:
WARNING:absl:Need to use a loss function that computes losses per sample, ex: replace losses.mean_squared_error with tf.math.squared_difference. Invalid value passed for `per_example_loss`. Expected a tensor tensor with at least rank 1
Should just be a 1-line swap to fix, unless there's something I'm missing (though this seems minor enough to not need an issue/discussion first).
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).
For more information, open the CLA check for this pull request.
Made the recommended changes to be more general, and tested that it works for my use case, though noting that, right now, the map_structure won't make much of a difference, as the local MSE function will only be called when the action is a single continuous value (i.e. len(action_spec) == 1).
Though we may want to investigate if the current implementation now admits multiple continuous actions and remove the single continuous check in __init__
Closing this as part of cleanup. I don't see this getting to the finish after over a year. Not the best outcome; and just being honest. I am merging some of the other PRs where I can resolve the issues in the next day or so.