It seems that the importance sampling code part is wrong.

[yhy258]

https://github.com/ikostrikov/pytorch-trpo/blob/e200eb8a23b3c7941a0091efb9750dafa4b23cbb/main.py#L108-L119

The fixed log prob part of the line and the "get_loss" function part are exactly the same. The two parts are executed consecutively so that the two values ("fixed_log_prob", "log_prob") ​​are exactly the same. Is there a reason you wrote the code like this?

[asyua-ye]

get_kl，also has this problem