divergence issue

[ozgurgundogan]

When the number of layers has been increased like 2 conv layer, the network diverges and no matter what the learning rate is. I also added learning rate decay but it diverged again. I think there is another problem somewhere.