more details / guidance in need

[godlockin]

1. it's quite helpful to have a requirements.txt along with the project as i met some issues during i tried the model, e.g. i didn't know i need to install torch, numpy by pip, tronsformers by pip+git
2. it's quite challenge to setup the project

section I:

args.RUN_DEVICE = 'cpu' # line 7
args.MODEL_NAME = '$PATH/RWKV-4b-Pile-171M-20230202-7922' # line 17

== >

python chat.py
...

Run prompt...
Traceback (most recent call last):
  File "chat.py", line 216, in <module>
    out = run_rnn(tokenizer.tokenizer.encode(init_prompt))
  File "chat.py", line 184, in run_rnn
    current_state = model.forward(model_tokens, current_state, preprocess_only = True)
  File "$PATH/ChatRWKV/src/model_run.py", line 191, in forward
    state = torch.zeros(args.n_layer * 5, args.n_embd, device=self.RUN_DEVICE)
AttributeError: 'types.SimpleNamespace' object has no attribute 'n_layer'

section II:

if '-1B5-' in args.MODEL_NAME or '/1.5-' in args.MODEL_NAME:
    args.n_layer = 24
    args.n_embd = 2048
elif '-3B-' in args.MODEL_NAME or '/3-' in args.MODEL_NAME:
    args.n_layer = 32
    args.n_embd = 2560
elif '-7B-' in args.MODEL_NAME or '/7-' in args.MODEL_NAME:
    args.n_layer = 32
    args.n_embd = 4096
elif '-14B-' in args.MODEL_NAME or '/14-' in args.MODEL_NAME:
    args.n_layer = 40
    args.n_embd = 5120
else: # line 41
    args.n_layer = 24
    args.n_embd = 768

==>

Run prompt...
Traceback (most recent call last):
  File "chat.py", line 219, in <module>
    out = run_rnn(tokenizer.tokenizer.encode(init_prompt))
  File "chat.py", line 187, in run_rnn
    current_state = model.forward(model_tokens, current_state, preprocess_only = True)
  File "$PATH/ChatRWKV/src/model_run.py", line 197, in forward
    x = self.LN(x, w.blocks[i].ln0)
  File "$PATH/ChatRWKV/src/model_run.py", line 103, in LN
    return F.layer_norm(x, (self.args.n_embd,), weight=w.weight, bias=w.bias)
  File "/usr/local/Caskroom/miniconda/base/envs/chatbot/lib/python3.8/site-packages/torch/nn/functional.py", line 2515, in layer_norm
    return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'

section III

args.FLOAT_MODE = 'fp32' # line 8

finally, it works

[BlinkDL]

1. Hi you can pip install transformers too.
2. v4a and v4b models are NOT compatible with the code here (which is only for v4) and you will get bad results. Use v4 models.