dfdx icon indicating copy to clipboard operation
dfdx copied to clipboard
verified publisher icon coreylowman

Added draft BatchNorm1D implementation

Open vikigenius opened this issue 3 years ago • 2 comments

This is a WIP draft PR for batch normalization, wanted to discuss some potential design choices and issues.

vikigenius avatar Aug 21 '22 00:08 vikigenius

Since the behavior of batchnorm is different for training and inference we need some way to test for both.

One possibility is to add a test that calls forward_mut multiple times and then a final forward call that uses the running statistics.

vikigenius avatar Aug 21 '22 00:08 vikigenius

Also, I have been pretty busy recently, so I would probably wait for https://github.com/coreylowman/dfdx/issues/163 before taking this on again.

vikigenius avatar Sep 07 '22 16:09 vikigenius