Danijar Hafner
Danijar Hafner
I was able to fix the same issue by getting rid of the `\n` that was unnecessary since I already filled the whole line.
Dry run would be very useful in my opinion, too.
Thank for you investigating this! Computing returns on the CPU seems reasonable anyway. Would you like to create a PR with this change? I don't know of any more efficient...
Haha, okay. It seems like there is a problem when running the `tf.scan()` or `tf.reverse()` ops on GPU that are used to compute the return. Placing this loop on CPU...
Yes, this seems reasonable. Did you train an agent like this to see if it affects performance?
It could be normal that TensorBoard doesn't show anything for a while. The frequency for writing logs is define inside `_define_loop()` in [train.py](https://github.com/tensorflow/agents/blob/master/agents/scripts/train.py#L59). This is set to twice per epoch...
Please wrap code blocks in 3 back ticks. Your configuration must be pickable and it looks like yours is not. Try to define it without using lambdas. As alternatives, define...
Where is the `env` defined in your config? You should not create the environments in the main process as you did by removing the lambda.
No, I meant you should undo the change you made to the batch env line. You define environments in your config by setting `env = ...` to either the name...