ggml icon indicating copy to clipboard operation
ggml copied to clipboard

feat: implement backward computation for more operators

Open Ronsor opened this issue 6 months ago • 5 comments

This PR will add backward computations for most operators once completed.

  • [x] Tanh
  • [x] Sigmoid
  • [x] GELU + GELU (quick)
  • [x] ELU
  • [x] clamp
  • [x] LeakyReLU
  • [x] mean
  • [x] concat

Leaving pad, im2col, and norm for a future PR now.

Currently unsure if I should fuse the multiply + gradient computation for gelu_back/gelu_quick_back like with silu_back.

Ronsor avatar Aug 12 '24 20:08 Ronsor