ggml
ggml copied to clipboard
feat: implement backward computation for more operators
This PR will add backward computations for most operators once completed.
- [x] Tanh
- [x] Sigmoid
- [x] GELU + GELU (quick)
- [x] ELU
- [x] clamp
- [x] LeakyReLU
- [x] mean
- [x] concat
Leaving pad
, im2col
, and norm
for a future PR now.
Currently unsure if I should fuse the multiply + gradient computation for gelu_back
/gelu_quick_back
like with silu_back
.