loss going to nan

[sohamghosh121]

I am trying to use this for a dataset of my own - VGG-fc8 features and GLoVE embeddings. However, when I add a regularization parameter, the loss quickly goes to nan. If I remove it, the loss grows larger (from negative to close to 0). What might I be doing wrong?