lightning-thunder icon indicating copy to clipboard operation
lightning-thunder copied to clipboard
verified publisher icon Lightning-AI

Support batch_norm backward and nvfuser (PR2278)

Open kiya00 opened this issue 1 year ago • 1 comments

Support batch norm:

  • Add backward support

  • Add nvFuser support

  • Type promotion of input/weight/bias are handled in thunder

  • Running stats are left as-is, and let nvfuserex/torchex to take care of in-place updating

kiya00 avatar Mar 21 '24 09:03 kiya00