Lasse Espeholt

Using Python grpc would be significantly slower than C++ and the custom batching.

you can still compile grpc inside docker, copy the file and then not use docker at all when you run the training.

@zhuliwen thanks!

" just adapted script a little according to #14 and it was built successfully and passed tests. "

How did you make it build on tf-nightly? Try and change the base image of Docker.grpc to nightly-custom-op-gpu-ubuntu16

It seems to work for me: Use your docker file for Dockerfile.grpc ./grpc/build.sh (to update grpc_cc.so and service_pb2.py) Add RUN pip3 install tf-nightly==2.5.0.dev20201025 in e.g. Dockerfile.atari ./run_local.sh atari r2d2 4

The bit packing code is used to make CPUAccelerator transfers faster. For this particular code, we have the following in the documentation: frames: [time, batch_size, height, width, channels]. These should...

Looks sensible (see tests in networks_test.py) although frame stacking can be done quite simple when specialized tricks aren't used. Instead of using a tensor with all frames, one can do...

One needs to do something similar to this part of the code to run with multiple GPUs. https://github.com/google-research/seed_rl/blob/eff7aaa7ab5843547fbf383fcc747b7a8ca67632/common/utils.py#L42-L52 Can you show me the exact change you did to the code?

Can you try: def apply_gradients(_): optimizer.apply_gradients(zip([g + 0 for g in temp_grads], agent.trainable_variables)) or def apply_gradients(_): optimizer.apply_gradients(zip([g.read_value() for g in temp_grads], agent.trainable_variables))