RWKV-LM icon indicating copy to clipboard operation
RWKV-LM copied to clipboard

Where's the cuda backward function for v7?

Open bmilde opened this issue 1 year ago • 1 comments
trafficstars

As stated in the title, seems cuda_backward and kernel_backward is missing in RWKV-v7/cuda/wkv7.cu and RWKV-v7/cuda/wkv7_op.cpp

If you try to train a model with RWKV-v7 I'm getting:

NotImplementedError: You must implement either the backward or vjp method for your custom autograd.Function to use it with backward mode AD.

bmilde avatar Oct 03 '24 16:10 bmilde