It seems training not working(rewards don't converge at all)

[zhiyiZeng]

This repo is pretty awesome. I'm trying to run a basic demo, but the training process seems not working at all (rewards don't converge at all). However, the agent still outperforms B&H a lot.(even when the reward is negative! ) I'm confused by this situation. Is there an explanation about this?

The graph is rewards with training epochs=50.