RWKV-LM icon indicating copy to clipboard operation
RWKV-LM copied to clipboard

Published 20 hours ago • verified publisher icon BlinkDL

Implementation details about wkv

Open kxzxvbk opened this issue 1 year ago • 0 comments

Amazing work! But I'm really confused about the implementation details for wkv cuda kernel (in RWKV-LM\RWKV-v4neo\cuda\wkv_cuda.cu). How does the implementation match the equations shown in README? Could you please give a more detailed comment about it? For example, what is the meaning of local variable p, pp, ... Thanks

kxzxvbk avatar Mar 07 '23 15:03 kxzxvbk