Kian Kim

Thanks for the quick reply. I've checked the repo again, and I still only see code for running random search with ASHA. I'm mentioning this because the paper claims the...

@JaminFong I've looked at your implementation and only found instructions for running the 2nd order version of the algorithm. Could you specify the instructions on running the algorithm on just...

That's the normalization for the output of the projection head. I was asking about the normalization of the output of the encoder, which is a different thing.

Thanks for your quick reply. Any further clarification on this would be super helpful!

I'm replying via email because I could not find your question in the Github issue thread. The normalization of the output of the encoder network is described under Section 3.1...

Sure. The normalization of the output of the encoder network is described under Section 3.1 Page 5 under the "An encoder network" bullet point. It is mentioned as "...This representation...

((std - 4).clamp(min=0) ** 2).mean() shouldn't this be ((0.25*std - 1).clamp(min=0) ** 2).mean()? The former is basically around 16 times the latter because the thing that your squaring is multiplied...

> @clefourrier has been investigating the [DROP benchmark](https://aclanthology.org/N19-1246/) and has discovered that it has some serious problems, as are detailed in our recent blog post together and her previous this...