Using Custom NN - 'couldn't compute a good step' with one NN path set to non-trainable

[ryanmaxwell96]

Hello,

I've been trying to use my own customized NN architecture to train Ant-V2. I have two paths that look like: 64->64->64->4 inputs (27) -> 64->64->64->4

where the top 4 outputs refer to the front (on the video, the right) legs and the bottom 4 outputs refer to the back (left on video) legs. Now, for the first training session, I'd like to lock the bottom path of the neural network to prevent it from training (I have locked these legs in place so that they cannot move for the first stage of training). I've tried by setting this path to trainable=False but when I do this, I get the error: couldn't compute a good step. (I'm using trpo_mpi.py alg) Does anyone know of a reason why setting a portion of the NN to trainable=False would kill the entire thing? I would have thought that the trainable path should still be able to function.