fwdgrad
fwdgrad copied to clipboard
orobix
No speed-up in my implementation too
I implemented this papaer with torch.autograd.forward_ad. However, fwd gradient showed no speed-up compared to fwd+bwd.
It would be interesting for us to see your implementation as well. If you want, you can make a PR to our repo with your code. So we can have multiple implementations available.
I ran the code from the repository, but I couldn't replicate the results mentioned in the paper, especially regarding the CNN. I used the hyperparameter settings specified in the paper.
May I inquire if there are alternative parameter settings available?
Hi, unfortunately we weren't able to reproduce the same results too. The hyperparameters we used are the same reported in the paper, but we don't now if alternative hyperparameters settings are available.
We believe that the difference between our implementation and the official one are due to the fact that they did not use functorch