Somshubra Majumdar

@tchaton I've refactored the codebase to allow extending a base wrapper called `OptimizerWrapper` which will handle most of the heavy lifting. Now, `NormalizedOptimizer` and `ClippedOptimizer` extend that and we can...

That looks like some great work, but this wrapper is over an Optimizer, and not a Layer. Therefore I don't think it's support can be included in a similar manner.

Did the paper update recently? The beta parameter wasn't there when I implemented this. In any case, it doesn't sound like a complicated change. If you'd like, you could submit...

You're right that it's an easy fix, but it's temporary. I will have to look into it closely to find all the places where the data format can cause this...

@jaretburkett since you have suggested the fix, it would be correct for you to add the PR. However if you don't want that, then I'll take care of it.

This issue is due to the fact that when I first wrote this repo, I was using Theano with channels_first data format. However, recent updates were made to use Tensorflow...

@jaretburkett Thanks for the thorough reply. I'll look into the reshape flag in weight loading, but I think it's more for weights loaded in Keras 2.x. However, I'm happy to...

87.76% for BN vs 87.26% for SN. This is probably due to 0.98 momentum on SN which is different from the papers recommended value of 0.997, vs the 0.99 default...

Probably more to do with the batch size. The maximum batch size they use is 32 I believe. I use batch size 200 for both the models.

Embedding lookup is done for the Keras Embedding layer, which I don't use in this model. It is likely an issue in your overall architecure. Embedding layers accept only integer...