The output layer should not using RELU activation function.

[wanjunhong0]

tf.contrib.layers.fully_connected (activation_fn: Activation function. The default value is a ReLU function. Explicitly set it to None to skip it and maintain a linear activation.)

In the original paper, the last FC layer is set to linear layer. So the in the dpn.py, self.predictions = tf.contrib.layers.fully_connected(fc1, len(VALID_ACTIONS)) should be set to self.predictions = tf.contrib.layers.fully_connected(fc1, len(VALID_ACTIONS), activation_fn=None)