How to write the RWKV in autogressive style like RNN

[HaiFengZeng]

instead of running in a teacher-forcing in training stage, can we run the model in free running style?:

for i in range(T):
    X_{t}=RNN(X_{t-1},hidden_state)

since teacher-forcing may have an exposure bias problem.

[BlinkDL]

yes you can

[BlinkDL]

see https://github.com/RWKV/RWKV-infctx-trainer