agents icon indicating copy to clipboard operation
agents copied to clipboard

Switch tf.losses.mse with tf.math.squared_difference in BehavioralCloningAgent

Open egordon opened this issue 3 years ago • 2 comments

When training a BC Agent with a continuous action space, I received the following warning:

WARNING:absl:Need to use a loss function that computes losses per sample, ex: replace losses.mean_squared_error with tf.math.squared_difference. Invalid value passed for `per_example_loss`. Expected a tensor tensor with at least rank 1

Should just be a 1-line swap to fix, unless there's something I'm missing (though this seems minor enough to not need an issue/discussion first).

egordon avatar Feb 25 '22 23:02 egordon

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

For more information, open the CLA check for this pull request.

google-cla[bot] avatar Feb 25 '22 23:02 google-cla[bot]

Made the recommended changes to be more general, and tested that it works for my use case, though noting that, right now, the map_structure won't make much of a difference, as the local MSE function will only be called when the action is a single continuous value (i.e. len(action_spec) == 1).

Though we may want to investigate if the current implementation now admits multiple continuous actions and remove the single continuous check in __init__

egordon avatar Mar 02 '22 19:03 egordon

Closing this as part of cleanup. I don't see this getting to the finish after over a year. Not the best outcome; and just being honest. I am merging some of the other PRs where I can resolve the issues in the next day or so.

tfboyd avatar Sep 30 '23 13:09 tfboyd